Mapache.PromptEvalDotNet 0.0.2

.NET 8.0

dotnet add package Mapache.PromptEvalDotNet --version 0.0.2

NuGet\Install-Package Mapache.PromptEvalDotNet -Version 0.0.2

This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.

<PackageReference Include="Mapache.PromptEvalDotNet" Version="0.0.2" />

For projects that support PackageReference, copy this XML node into the project file to reference the package.

paket add Mapache.PromptEvalDotNet --version 0.0.2

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

#r "nuget: Mapache.PromptEvalDotNet, 0.0.2"

#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.

// Install Mapache.PromptEvalDotNet as a Cake Addin
#addin nuget:?package=Mapache.PromptEvalDotNet&version=0.0.2

// Install Mapache.PromptEvalDotNet as a Cake Tool
#tool nuget:?package=Mapache.PromptEvalDotNet&version=0.0.2

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Mapache.PromptEvalDotNet NuGet Package

This NuGet package, Mapache.PromptEvalDotNet, empowers your .NET projects with a robust and versatile library for evaluating, testing, and validating applications and systems using Large Language Models (LLMs). Seamlessly integrate AI/ML evaluations into your projects to assess performance, accuracy, and robustness with ease.

Installation

Install the Mapache.PromptEvalDotNet NuGet package via the NuGet Package Manager or the .NET CLI:

dotnet add package Mapache.PromptEvalDotNet

Usage

Sample Code

// Sample code for using Mapache.PromptEvalDotNet
var kernelBuilder = Kernel.CreateBuilder();

kernelBuilder.AddAzureOpenAIChatCompletion(config["AZURE_OPENAI_MODEL"]!, config["AZURE_OPENAI_ENDPOINT"]!, config["AZURE_OPENAI_KEY"]!);

var evalFactory = new Mapache.PromptEvalDotNet.EvaluatorFactory("MyProjectEvals", builder.Build());

var coherence = await evalFactory.Coherence("blog-posts").GetScore(prompt, answer);

Explanation

Kernel Setup: Begin by setting up the kernel using Kernel.CreateBuilder().
Integration with Azure OpenAI Chat Completion: Use kernelBuilder.AddAzureOpenAIChatCompletion to integrate with Azure OpenAI Chat Completion, providing the necessary configuration parameters.
Evaluator Factory Initialization: Create an instance of Mapache.PromptEvalDotNet.EvaluatorFactory by providing a unique project name and the built kernel.
Evaluate Coherence: Utilize the evalFactory.Coherence method to evaluate coherence for a specific context, such as "blog-posts."
Get Score: Obtain the coherence score by calling GetScore with the prompt and answer.

Contributing

See the project repo for more information.

Product	Compatible and additional computed target framework versions.
.NET	net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed.