GovUK.Dfe.AI.Agents.Evaluation 0.0.1-prerelease.394

Prefix Reserved
.NET 10.0 This package targets .NET 10.0. The package is compatible with this framework or higher.
This is a prerelease version of GovUK.Dfe.AI.Agents.Evaluation. 
dotnet add package GovUK.Dfe.AI.Agents.Evaluation --version 0.0.1-prerelease.394
                    


                    

                
NuGet\Install-Package GovUK.Dfe.AI.Agents.Evaluation -Version 0.0.1-prerelease.394
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package. 
<PackageReference Include="GovUK.Dfe.AI.Agents.Evaluation" Version="0.0.1-prerelease.394" />
For projects that support PackageReference, copy this XML node into the project file to reference the package. 
<PackageVersion Include="GovUK.Dfe.AI.Agents.Evaluation" Version="0.0.1-prerelease.394" />
                    


                            Directory.Packages.props
                        

                    

                
<PackageReference Include="GovUK.Dfe.AI.Agents.Evaluation" />
                    


                            Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package. 
paket add GovUK.Dfe.AI.Agents.Evaluation --version 0.0.1-prerelease.394
                    


                    

                
#r "nuget: GovUK.Dfe.AI.Agents.Evaluation, 0.0.1-prerelease.394"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package. 
#:package GovUK.Dfe.AI.Agents.Evaluation@0.0.1-prerelease.394
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package. 
#addin nuget:?package=GovUK.Dfe.AI.Agents.Evaluation&version=0.0.1-prerelease.394&prerelease
                    


                            Install as a Cake Addin
                        

                    

                
#tool nuget:?package=GovUK.Dfe.AI.Agents.Evaluation&version=0.0.1-prerelease.394&prerelease
                    


                            Install as a Cake Tool

GovUK.Dfe.AI.Agents.Evaluation

A judge model scores your GovUK.Dfe.AI.Agents agents' answers from 1 to 5 for:

  • Groundedness: is the answer supported by the evidence?
  • Relevance: does it answer the prompt?

Use the scores to spot quality dropping after a prompt or model change, and to stop a worse agent in your release gate.

Set up

dotnet add package GovUK.Dfe.AI.Agents.Evaluation

"AiAgents": {
  "Evaluation": {
    "JudgeModel": "<connection>/gpt-5.1",
    "SampleRate": 0.05
  }
}

builder.Services.AddAgents(builder.Configuration, agents => agents
    .AddAgents(BriefingAgents.All)
    .AddQualityEvaluation());
Setting Default Purpose
JudgeModel Required Any model in your Foundry project. It's called with the app's Foundry credential
SampleRate 0.05 Share of live runs scored, from 0 to 1. 0 scores only release-gate tests

A missing JudgeModel or a SampleRate outside 0 to 1 fails startup, listed with any other missing settings.

What gets scored

  • Live runs: a SampleRate share is scored in the background, so runs aren't slowed. If the judge falls behind, extra samples are dropped.
  • Release gate: every test case is scored, so report.FailuresAgainst(gate, baseline) can fail the build. See Release gate.

Metrics

Recorded under AgentTelemetry.SourceName, so the core package's OpenTelemetry setup already collects them.

Metric Records
dfe.ai_agents.evaluation.score Each score, by agent, version and gen_ai.evaluation.name
dfe.ai_agents.evaluation.samples_dropped Sampled runs not scored because the judge fell behind. If this rises, lower SampleRate

Good to know

  • Cost: each scored answer costs one judge call.
  • Nothing stored: like agent runs, judge calls store nothing in Foundry, though they carry the run's prompt, evidence and answer.
  • Reasoning models work as judges: the judge sends only the model name and messages.
  • Missing scores: if a metric can't be scored (e.g. a wrong model name), the reason is logged.
  • Custom metrics: agents.AddQualityEvaluation(evaluator: new MyEvaluator()) with any Microsoft.Extensions.AI IEvaluator.
  • Your own scorer: agents.AddCustomQualityEvaluation(sp => new MyEvaluator()) with any IAgentRunEvaluator; no JudgeModel needed, and SampleRate still applies.
  • No safety metrics (Microsoft's are preview-only): use the Guardrails package to block harmful content.
Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.0.1-prerelease.394 0 10/14/2026
0.0.1-prerelease.363 0 10/14/2026
0.0.1-prerelease.333 31 10/13/2026
0.0.1-prerelease.305 45 10/9/2026
0.0.1-prerelease.278 41 10/9/2026
0.0.1-prerelease.252 39 10/9/2026
0.0.1-prerelease.227 43 10/9/2026
0.0.1-prerelease.203 41 10/9/2026
0.0.1-prerelease.180 40 10/9/2026
0.0.1-prerelease.158 42 10/9/2026
0.0.1-prerelease.137 44 10/9/2026
0.0.1-prerelease.117 44 10/9/2026
0.0.1-prerelease.98 51 10/7/2026
0.0.1-prerelease.80 46 10/7/2026
0.0.1-prerelease.63 52 10/6/2026
0.0.1-prerelease.47 55 10/5/2026
0.0.1-prerelease.33 55 10/5/2026
0.0.1-prerelease.20 57 10/2/2026
0.0.1-prerelease.10 56 10/2/2026