This is the multi-page printable view of this section. Click here to print.
With the Dapr AI package, you can interact with the Dapr AI workloads from a .NET application.
Today, Dapr provides the Conversational API to engage with large language models. To get started with this workload, walk through the Dapr Conversational AI how-to guide.
The Dapr AI client package allows you to interact with the AI capabilities provided by the Dapr sidecar.
A DaprConversationClient is a version of the Dapr client that is dedicated to interacting with the Dapr Conversation
API. It can be registered alongside a DaprClient and other Dapr clients without issue.
It maintains access to networking resources in the form of TCP sockets used to communicate with the Dapr sidecar.
For best performance, create a single long-lived instance of DaprConversationClient and provide access to that shared
instance throughout your application. DaprConversationClient instances are thread-safe and intended to be shared.
This can be aided by utilizing the dependency injection functionality. The registration method supports registration
as a singleton, a scoped instance or as transient (meaning it’s recreated every time it’s injected), but also enables
registration to utilize values from an IConfiguration or other injected services in a way that’s impractical when
creating the client from scratch in each of your classes.
Avoid creating a DaprConversationClient for each operation.
A DaprConversationClient can be configured by invoking methods on the DaprConversationClientBuilder class before
calling .Build() to create the client itself. The settings for each DaprConversationClient are separate
and cannot be changed after calling .Build().
var daprConversationClient = new DaprConversationClientBuilder()
.UseDaprApiToken("abc123") // Specify the API token used to authenticate to other Dapr sidecars
.Build();
The DaprConversationClientBuilder contains settings for:
JsonSerializerOptions object used to configure JSON serializationGrpcChannelOptions object used to configure gRPCHttpClient instance used by the SDKHttpClient instance when making requests to the sidecarThe SDK will read the following environment variables to configure the default values:
DAPR_HTTP_ENDPOINT: used to find the HTTP endpoint of the Dapr sidecar, example: https://dapr-api.mycompany.comDAPR_GRPC_ENDPOINT: used to find the gRPC endpoint of the Dapr sidecar, example: https://dapr-grpc-api.mycompany.comDAPR_HTTP_PORT: if DAPR_HTTP_ENDPOINT is not set, this is used to find the HTTP local endpoint of the Dapr sidecarDAPR_GRPC_PORT: if DAPR_GRPC_ENDPOINT is not set, this is used to find the gRPC local endpoint of the Dapr sidecarDAPR_API_TOKEN: used to set the API tokenDapr’s use of CancellationToken for cancellation relies on the configuration of the gRPC channel options. If you need
to configure these options yourself, make sure to enable the ThrowOperationCanceledOnCancellation setting.
var daprConversationClient = new DaprConversationClientBuilder()
.UseGrpcChannelOptions(new GrpcChannelOptions { ... ThrowOperationCanceledOnCancellation = true })
.Build();
DaprConversationClientThe APIs on DaprConversationClient perform asynchronous operations and accept an optional CancellationToken parameter. This
follows a standard .NET practice for cancellable operations. Note that when cancellation occurs, there is no guarantee that
the remote endpoint stops processing the request, only that the client has stopped waiting for completion.
When an operation is cancelled, it will throw an OperationCancelledException.
DaprConversationClient via dependency injectionUsing the built-in extension methods for registering the DaprConversationClient in a dependency injection container can
provide the benefit of registering the long-lived service a single time, centralize complex configuration and improve
performance by ensuring similarly long-lived resources are re-purposed when possible (e.g. HttpClient instances).
There are three overloads available to give the developer the greatest flexibility in configuring the client for their
scenario. Each of these will register the IHttpClientFactory on your behalf if not already registered, and configure
the DaprConversationClientBuilder to use it when creating the HttpClient instance in order to re-use the same instance as
much as possible and avoid socket exhaustion and other issues.
In the first approach, there’s no configuration done by the developer and the DaprConversationClient is configured with the
default settings.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddDaprConversationClient(); //Registers the `DaprConversationClient` to be injected as needed
var app = builder.Build();
Sometimes the developer will need to configure the created client using the various configuration options detailed
above. This is done through an overload that passes in the DaprConversationClientBuiler and exposes methods for configuring
the necessary options.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddDaprConversationClient((_, daprConversationClientBuilder) => {
//Set the API token
daprConversationClientBuilder.UseDaprApiToken("abc123");
//Specify a non-standard HTTP endpoint
daprConversationClientBuilder.UseHttpEndpoint("http://dapr.my-company.com");
});
var app = builder.Build();
Finally, it’s possible that the developer may need to retrieve information from another service in order to populate
these configuration values. That value may be provided from a DaprClient instance, a vendor-specific SDK or some
local service, but as long as it’s also registered in DI, it can be injected into this configuration operation via the
last overload:
var builder = WebApplication.CreateBuilder(args);
//Register a fictional service that retrieves secrets from somewhere
builder.Services.AddSingleton<SecretService>();
builder.Services.AddDaprConversationClient((serviceProvider, daprConversationClientBuilder) => {
//Retrieve an instance of the `SecretService` from the service provider
var secretService = serviceProvider.GetRequiredService<SecretService>();
var daprApiToken = secretService.GetSecret("DaprApiToken").Value;
//Configure the `DaprConversationClientBuilder`
daprConversationClientBuilder.UseDaprApiToken(daprApiToken);
});
var app = builder.Build();
To get started with the Dapr AI .NET SDK client, install the Dapr.AI package from NuGet:
dotnet add package Dapr.AI
A DaprConversationClient maintains access to networking resources in the form of TCP sockets used to communicate with the Dapr sidecar.
The AddDaprAiConversation() method will register the Dapr client ASP.NET Core dependency injection and is the recommended approach
for using this package. This method accepts an optional options delegate for configuring the DaprConversationClient and a
ServiceLifetime argument, allowing you to specify a different lifetime for the registered services instead of the default Singleton
value.
The following example assumes all default values are acceptable and is sufficient to register the DaprConversationClient:
services.AddDaprAiConversation();
The optional configuration delegate is used to configure the DaprConversationClient by specifying options on the
DaprConversationClientBuilder as in the following example:
services.AddSingleton<DefaultOptionsProvider>();
services.AddDaprAiConversation((serviceProvider, clientBuilder) => {
//Inject a service to source a value from
var optionsProvider = serviceProvider.GetRequiredService<DefaultOptionsProvider>();
var standardTimeout = optionsProvider.GetStandardTimeout();
//Configure the value on the client builder
clientBuilder.UseTimeout(standardTimeout);
});
Rather than using dependency injection, a DaprConversationClient can also be built using the static client builder.
For best performance, create a single long-lived instance of DaprConversationClient and provide access to that shared instance throughout
your application. DaprConversationClient instances are thread-safe and intended to be shared.
Avoid creating a DaprConversationClient per-operation.
A DaprConversationClient can be configured by invoking methods on the DaprConversationClientBuilder class before calling .Build()
to create the client. The settings for each DaprConversationClient are separate and cannot be changed after calling .Build().
var daprConversationClient = new DaprConversationClientBuilder()
.UseJsonSerializerSettings( ... ) //Configure JSON serializer
.Build();
See the .NET documentation here for more information about the options available when configuring the Dapr client via the builder.
Put the Dapr AI .NET SDK to the test. Walk through the samples to see Dapr in action:
| SDK Samples | Description |
|---|---|
| SDK samples | Clone the SDK repo to try out some examples and get started. |
This part of the .NET SDK allows you to interface with the Conversations API to send and receive messages from large language models.
To get started with this SDK, install both the Dapr.AI and Dapr.AI.Microsoft.Extensions packages from NuGet:
dotnet add package Dapr.AI
dotnet add package Dapr.AI.Microsoft.Extensions
The DaprChatClient is a Dapr-based implementation of the IChatClient interface provided in the
Microsoft.Extensions.AI.Abstractions package using Dapr’s [conversation building block]({{ ref conversation-overview.md }}). It allows
developers to build against the types provided by Microsoft’s abstraction while providing the greatest conformity to the
Dapr conversation building block available. As both approaches adopt OpenAI’s API approach, these are expected to increasingly
converge over time.
The Dapr.AI.Microsoft.Extensions package implements the Microsoft.Extensions.AI abstractions, providing a unified API for
AI services in .NET applications. Microsoft.Extensions.AI is designed to offer a consistent programming model across
different AI providers and scenarios. For detailed information about Microsoft.Extensions.AI, refer to the
official documentation.
The DaprChatClient can be registered with the dependency injection container using several extension methods. First,
ensure that you reigster the DaprConversationClient that’s part of the Dapr.AI package from NuGet:
services.AddDaprConversationClient();
Then register the DaprChatClient with your conversation component name:
services.AddDaprChatClient("my-conversation-component");
You can confiugre the DaprChatClient using the DaprChatClientOptions though the current implementation only
provides configuration for the component name itself. This is expected to change in future releases.
services.AddDaprChatClient("my-conversation-component", options =>
{
// Configure additional options here
});
You can also configure the service lifetime (this defaults to ServiceLifetime.Scoped):
services.AddDaprChatClient("my-conversation-component", ServiceLifetime.Singleton);
Once registered, you can inject and use IChatClient in your services:
public class ChatService(IChatClient chatClient)
{
public async Task<IReadOnlyList<string>> GetResponseAsync(string message)
{
var response = await chatClient.GetResponseAsync([
new ChatMessage(ChatRole.User,
"Please write me a poem in iambic pentameter about the joys of using Dapr to develop distributed applications with .NET")
]);
return response.Messages.Select(msg => msg.Text).ToList();
}
}
The DaprChatClient does not yet support streaming responses and use of the corresponding GetStreamingResponseAsync
methods will throw a NotImplemenetedException. This is expected to change in a future release once the Dapr runtime
supports this functionality.
The client supports function calling through the Microsoft.Extensions.AI tool integration. Tools registered with the
conversation will be automatically available to the large language model.
string GetCurrentWeather() => Random.Shared.NextDouble() > 0.5 ? "It's sunny today!" : "It's raining today!";
var toolChatOptions = new ChatOptions { Tools = [AIFunctionFactory.Create(GetCurrentWeather, "weather")] };
var toolResponse = await chatClient.GetResponseAsync("What's the weather like today?", toolChatOptions);
foreach (var toolResp in toolResponse.Messages)
{
Console.WriteLine(toolResp);
}
The DaprChatClient integrates with Dapr’s error handling and will throw appropriate exceptions when issues occur.
The underlying Dapr conversation component can be configured with metadata and parameters through the Dapr conversation
building block configuration. The DaprChatClient will respect these settings when making calls to the conversation component.
Service Lifetime: Use ServiceLifetime.Scoped or ServiceLifetime.Singleton for the DaprChatClient registration to avoid creating multiple instances unnecessarily.
Error Handling: Always wrap calls in appropriate try-catch blocks to handle both Dapr-specific and general exceptions.
Resource Management: The DaprChatClient properly implements IDisposable through its base classes, so resources are automatically managed when using dependency injection.
Configuration: Configure your Dapr conversation component properly to ensure optimal performance and reliability.