Mistral.SDK.Extended 3.0.3

Mistral.SDK.Extended

Mistral.SDK.Extended is an unofficial C# client designed for interacting with the Mistral API. It is a fork of Mistral.SDK with selected community fixes and additional features, targeting .NET 8 and .NET 10.

Table of Contents

• Installation
• API Keys
• HttpClient
• Usage
• Examples
  • Non-Streaming Call
  • Streaming Call
  • IChatClient
  • Structured Outputs
  • List Models
  • Embeddings
  • OCR
  • Function Calling
• Contributing
• License

Installation

Install Mistral.SDK.Extended via the NuGet package manager:

dotnet add package Mistral.SDK.Extended --version 3.0.3

Supported target frameworks: net8.0 and net10.0.

API Keys

You can load the API Key from an environment variable named MISTRAL_API_KEY by default. Alternatively, you can supply it as a string to the MistralClient constructor.

HttpClient

The MistralClient can optionally take a custom HttpClient in the MistralClient constructor, which allows you to control elements such as retries and timeouts. Note: If you provide your own HttpClient, you are responsible for disposal of that client.

Usage

There are three ways to start using the MistralClient. The first is to simply new up an instance of the MistralClient and start using it, the second is to use the messaging/Embedding client with the new Microsoft.Extensions.AI.Abstractions builder. The third is to use the Completions client with Microsoft.SemanticKernel. Brief examples of each are below.

Option 1:

var client = new MistralClient();

Option 2:

//chat client
IChatClient client = new MistralClient().AsIChatClient();

//embeddings generator
IEmbeddingGenerator<string, Embedding<float>> client = new MistralClient().Embeddings;

Option 3:

using Microsoft.SemanticKernel;

var skChatService =
    new ChatClientBuilder(new MistralClient().Completions)
        .UseFunctionInvocation()
        .Build()
        .AsChatCompletionService();


var sk = Kernel.CreateBuilder();
sk.Plugins.AddFromType<SkPlugins>("Weather");
sk.Services.AddSingleton<IChatCompletionService>(skChatService);

See integration tests for a more complete example.

All support all the core features of the MistralClient's Messaging and Embedding capabilities, but the latter will be fully featured in .NET 9 and provide built in telemetry and DI and make it easier to choose which SDK you are using.

Examples

Non-Streaming Call

Here's an example of a non-streaming call to the mistral-medium completions endpoint (other options are available and documented, but omitted for brevity):

var client = new MistralClient();
var request = new ChatCompletionRequest(
    //define model - required
    ModelDefinitions.MistralMedium,
    //define messages - required
    new List<ChatMessage>()
{
    new ChatMessage(ChatMessage.RoleEnum.System, 
        "You are an expert at writing sonnets."),
    new ChatMessage(ChatMessage.RoleEnum.User, 
        "Write me a sonnet about the Statue of Liberty.")
}, 
    //optional - defaults to false
    safePrompt: true, 
    //optional - defaults to 0.7
    temperature: 0, 
    //optional - defaults to null
    maxTokens: 500, 
    //optional - defaults to 1
    topP: 1, 
    //optional - defaults to null
    randomSeed: 32);
var response = await client.Completions.GetCompletionAsync(request);
Console.WriteLine(response.Choices.First().Message.Content);

Streaming Call

The following is an example of a streaming call to the mistral-medium completions endpoint:

var client = new MistralClient();
var request = new ChatCompletionRequest(
    ModelDefinitions.MistralMedium, 
    new List<ChatMessage>()
{
    new ChatMessage(ChatMessage.RoleEnum.System, 
        "You are an expert at writing sonnets."),
    new ChatMessage(ChatMessage.RoleEnum.User, 
        "Write me a sonnet about the Statue of Liberty.")
});
var results = new List<ChatCompletionResponse>();
await foreach (var res in client.Completions.StreamCompletionAsync(request))
{
    results.Add(res);
    Console.Write(res.Choices.First().Delta.Content);
}

IChatClient

The MistralClient has support for the new IChatClient from Microsoft and offers a slightly different mechanism for using the MistralClient. Below are a few examples.

//non-streaming
IChatClient client = new MistralClient().Completions;

var response = await client.GetResponseAsync(new List<ChatMessage>()
{
    new(ChatRole.System, "You are an expert at writing sonnets."),
    new(ChatRole.User, "Write me a sonnet about the Statue of Liberty.")
}, new() { ModelId = ModelDefinitions.OpenMistral7b });

Assert.IsTrue(!string.IsNullOrEmpty(response.Message.Text));

//streaming call
IChatClient client = new MistralClient().Completions;

var sb = new StringBuilder();
await foreach (var update in client.GetStreamingResponseAsync(new List<ChatMessage>()
    {
        new(ChatRole.System, "You are an expert at writing Json."),
        new(ChatRole.User, "Write me a simple 'hello world' statement in a json object with a single 'result' key.")
    }, new() { ModelId = ModelDefinitions.MistralLarge, ResponseFormat = ChatResponseFormat.Json }))
{
    sb.Append(update);
}

//parse json
Assert.IsNotNull(JsonSerializer.Deserialize<JsonResult>(sb.ToString()));

//Embeddings call
IEmbeddingGenerator<string, Embedding<float>> = new MistralClient().Embeddings;
            var response = await client.GenerateVectorAsync("hello world", new() { ModelId = ModelDefinitions.MistralEmbed });
            Assert.IsTrue(!response.IsEmpty);

//Functions call
IChatClient client = new MistralClient().Completions
    .AsBuilder()
    .UseFunctionInvocation()
    .Build();

ChatOptions options = new()
{
    ModelId = ModelDefinitions.MistralSmall,
    MaxOutputTokens = 512,
    ToolMode = ChatToolMode.Auto,
    Tools = [AIFunctionFactory.Create((string personName) => personName switch {
        "Alice" => "25",
        _ => "40"
    }, "GetPersonAge", "Gets the age of the person whose name is specified.")]
};

var res = await client.GetResponseAsync("How old is Alice?", options);

Assert.IsTrue(
    res.Message.Text?.Contains("25") is true,
    res.Message.Text);

Please see the integration tests for even more examples.

Structured Outputs

Custom JSON Schema responses are supported through the typed IChatClient API. Setting useJsonSchemaResponseFormat generates a schema for the result type, sends it to Mistral, and deserializes the response:

public sealed class CityResult
{
    public required string City { get; init; }
    public required string Country { get; init; }
}

IChatClient client = new MistralClient().Completions;

var response = await client.GetResponseAsync<CityResult>(
    [new(ChatRole.User, "Which country is Prague in?")],
    new ChatOptions
    {
        ModelId = ModelDefinitions.MistralLarge,
    },
    useJsonSchemaResponseFormat: true);

CityResult result = response.Result;
Console.WriteLine($"{result.City}, {result.Country}");

For a hand-authored schema or custom schema metadata, use the regular non-generic GetResponseAsync method and set ChatOptions.ResponseFormat to a value created with ChatResponseFormat.ForJsonSchema. The typed helper shown above generates its own schema.

List Models

The following is an example of a call to list the available models:

var client = new MistralClient();

var response = await client.Models.GetModelsAsync();

Embeddings

The following is an example of a call to the mistral-embed embeddings model/endpoint:

var client = new MistralClient();
var request = new EmbeddingRequest(
    ModelDefinitions.MistralEmbed, 
    new List<string>() { "Hello world" }, 
    EmbeddingRequest.EncodingFormatEnum.Float);
var response = await client.Embeddings.GetEmbeddingsAsync(request);

OCR

The OCR endpoint accepts public URLs and base64 data URIs for documents and images. Set the document type to document_url for PDFs and other documents, or image_url for images:

var client = new MistralClient();
var request = new OCRRequest
{
    Model = ModelDefinitions.MistralOCR,
    Document = new Document
    {
        Type = "document_url",
        DocumentUrl = "https://example.com/document.pdf",
    },
    IncludeImageBase64 = false,
};

var response = await client.OCR.GetOCRAsync(request);
Console.WriteLine(string.Join(Environment.NewLine, response.Pages.Select(page => page.Markdown)));

OCR can also be used through the standard IChatClient interface. The message must contain exactly one DataContent or UriContent. Since the OCR API is not streaming, its streaming IChatClient facade returns one update after processing completes.

var mistral = new MistralClient
{
    UseOcrEndpoint = true,
};
IChatClient client = mistral.AsIChatClient();

var message = new ChatMessage(ChatRole.User,
[
    new DataContent(await File.ReadAllBytesAsync("document.pdf"), "application/pdf"),
]);

ChatResponse response = await client.GetResponseAsync(
    [message],
    new ChatOptions { ModelId = ModelDefinitions.MistralOCR });

Console.WriteLine(response.Text);

Function Calling

The MistralClient supports Function Calling through a variety of mechanisms. It's worth noting that currently some models seem to hallucinate function calling behavior more than others, and this is a known issue with Mistral.

public enum TempType
{
    Fahrenheit,
    Celsius
}

[Function("This function returns the weather for a given location")]
public static async Task<string> GetWeather([FunctionParameter("Location of the weather", true)] string location,
    [FunctionParameter("Unit of temperature, celsius or fahrenheit", true)] TempType tempType)
{
    await Task.Yield();
    return "72 degrees and sunny";
}

//declared globally
var client = new MistralClient();
var messages = new List<ChatMessage>()
{
    new ChatMessage(ChatMessage.RoleEnum.User, "What is the weather in San Francisco, CA in Fahrenheit?")
};
var request = new ChatCompletionRequest(ModelDefinitions.MistralSmall, messages);
request.MaxTokens = 1024;
request.Temperature = 0.0m;
request.ToolChoice = ToolChoiceType.Auto;

request.Tools = Common.Tool.GetAllAvailableTools(includeDefaults: false, forceUpdate: true, clearCache: true).ToList();

var response = await client.Completions.GetCompletionAsync(request).ConfigureAwait(false);

messages.Add(response.Choices.First().Message);

foreach (var toolCall in response.ToolCalls)
{
    var resp = await toolCall.InvokeAsync<string>();
    messages.Add(new ChatMessage(toolCall, resp));
}

var finalResult = await client.Completions.GetCompletionAsync(request).ConfigureAwait(false);

Assert.IsTrue(finalResult.Choices.First().Message.Content.Contains("72"));

//from a func
var client = new MistralClient();
var messages = new List<ChatMessage>()
{
    new ChatMessage(ChatMessage.RoleEnum.User,"How many characters are in the word Christmas, multiply by 5, add 6, subtract 2, then divide by 2.1?")
};
var request = new ChatCompletionRequest(ModelDefinitions.MistralSmall, messages);

request.ToolChoice = ToolChoiceType.Auto;

request.Tools = new List<Common.Tool>
{
    Common.Tool.FromFunc("ChristmasMathFunction",
        ([FunctionParameter("word to start with", true)]string word,
            [FunctionParameter("number to multiply word count by", true)]int multiplier,
            [FunctionParameter("amount to add to word count", true)]int addition,
            [FunctionParameter("amount to subtract from word count", true)]int subtraction,
            [FunctionParameter("amount to divide word count by", true)]double divisor) =>
        {
            return ((word.Length * multiplier + addition - subtraction) / divisor).ToString(CultureInfo.InvariantCulture);
        }, "Function that can be used to determine the number of characters in a word combined with a mathematical formula")
};

var response = await client.Completions.GetCompletionAsync(request);

messages.Add(response.Choices.First().Message);

foreach (var toolCall in response.ToolCalls)
{
    var resp = toolCall.Invoke<string>();
    messages.Add(new ChatMessage(toolCall, resp));
}

var finalResult = await client.Completions.GetCompletionAsync(request);

Assert.IsTrue(finalResult.Choices.First().Message.Content.Contains("23"));

//see integration tests for examples like streaming function calls, calling a static or instance based function, and more.

Contributing

Pull requests are welcome with associated integration tests. If you're planning to make a major change, please open an issue first to discuss your proposed changes.

License

This project is licensed under the MIT License.