Detester 1.0.3

Detester is a .NET library that enables you to write deterministic tests for AI-powered applications. It provides a fluent builder API for testing AI responses, ensuring consistency and reliability in your AI integrations.

Using Azure OpenAI

using Azure.AI.OpenAI;
using Detester;
using Microsoft.Extensions.AI;
using System.ClientModel;

// Create Azure OpenAI client and wrap it as IChatClient
var azureClient = new AzureOpenAIClient(
    new Uri("https://your-resource.openai.azure.com"),
    new ApiKeyCredential("your-azure-api-key"));
var chatClient = azureClient.GetChatClient("gpt-4-deployment").AsIChatClient();

// Create a builder with the chat client
var builder = DetesterFactory.Create(chatClient);

// Or create a builder with ChatOptions (to configure tools, behavior, and settings)
var builder = DetesterFactory.Create(chatClient, new ChatOptions {/*setup*/});

// Execute a test
await builder
    .WithPrompt("Explain quantum computing in simple terms")
    .ShouldContainResponse("quantum")
    .AssertAsync();

Using Any IChatClient

Detester works with any IChatClient implementation from Microsoft.Extensions.AI:

using Detester;
using Microsoft.Extensions.AI;

// Use any IChatClient implementation (OpenAI, Azure OpenAI, Ollama, custom, etc.)
IChatClient chatClient = // your chat client implementation
var builder = DetesterFactory.Create(chatClient);

await builder
    .WithPrompt("Test prompt")
    .ShouldContainResponse("expected text")
    .AssertAsync();

Advanced Usage

Setting Model Instructions

Set custom instructions (system messages) to guide the model's behavior:

await new DetesterBuilder(chatClient)
    .WithInstruction("Answer concisely.")
    .WithPrompt("Summarize this text.")
    .ShouldContainResponse("summary")
    .ShouldContainAll("short", "summary")
    .ShouldContainAny("tl;dr", "summary", "in short")
    .ShouldNotContainResponse("error")
    .ShouldNotContain("stack trace")
    .ShouldNotContainAnyResponse("SECRET_KEY", "password")
    .ShouldMatchRegex(@"^[A-Z].+")
    .ShouldBeEqualTo("Exact match", StringComparison.OrdinalIgnoreCase) // if you want strict equality
    .AssertAsync();

Check the README file in the repo for more details: https://github.com/sa-es-ir/detester