Webority.Ai.Judgment 0.14.0

Webority.Ai

AI agents for Webority products on the Microsoft Agent Framework, over Azure OpenAI or Azure AI Foundry, published as NuGet. One factory builds every agent, and every agent it builds:

• retries a throttled or briefly unavailable model, a bounded number of times;
• refuses to run once the day's recorded spend reaches the cap, before anything is sent to the model;
• records who ran it, what it cost and every tool call it made, with tool arguments redacted unless declared safe;
• emits OpenTelemetry GenAI spans (agent, chat, one execute_tool per tool call), prompts off unless enabled.

The package never owns a DbContext. Its tables live in the product's context, created from the SQL the package ships. ApplyWeborityAi() maps the four core tables (usage events, model prices, tool calls, paused agent sessions); each add-on maps its own beside it: spend alerts (ApplyWeborityAiSpendAlerts()), chat (ApplyWeborityAiChat()), the judgment ledger (ApplyWeborityAiJudgment()) and the SQL knowledge store (ApplyWeborityAiKnowledge()).

Proprietary: for Webority internal use only (see LICENSE).

Packages

Wiring a product

// Startup.ConfigureServices
services.AddDbContext<AppDbContext>(...);
services.AddDbContextFactory<AppDbContext>(...);   // usage, prices and tool calls are written through their own contexts

services.AddWeborityAi<AppDbContext>(_configuration);
services.AddWeborityAiAzureOpenAi(_configuration);   // or AddWeborityAiFoundry or AddWeborityAiAnthropic: exactly one

// AppDbContext.OnModelCreating
modelBuilder.ApplyWeborityAi();

A host registers exactly one transport, the service that actually answers the model call. The chain above it (retry, spend cap, instrumentation, tool audit) is the same on both.

• Azure OpenAI v1 (AddWeborityAiAzureOpenAi, in the core): the OpenAI SDK's chat client against an Azure OpenAI resource's /openai/v1/ API. Pick it unless you need Foundry: the core has no preview dependency, so a product on it never rides a preview.

• Azure AI Foundry (AddWeborityAiFoundry, in Webority.Ai.Foundry): agents through a Foundry project. Pick it when the product uses a Foundry project's own capabilities. The package pulls the Foundry preview packages.

• Anthropic (AddWeborityAiAnthropic, in Webority.Ai.Anthropic): Claude deployments on an Azure AI Foundry resource, through Anthropic's SDK and its Foundry extension. Pick it for Claude models. Configure "Anthropic": { "ResourceName": "<resource>" } (the subdomain of https://<resource>.services.ai.azure.com); deployment names in Ai:Deployments are the Claude deployment names. It signs in with the same Entra credential as the other transports, for https://ai.azure.com/.default, so the identity needs access to the Foundry resource. Anthropic's 529 (overloaded) is retried like a 503. It answers no embeddings: IAiEmbeddingFactory throws NotSupportedException on such a host.

No transport stops the host at startup; a second one throws when it is registered.

Copy content/schema/AiUsageEvent.sql, AiModelPrice.sql, AiToolCall.sql and AiAgentSession.sql from the package into the product's Scripts/ (idempotent, guarded), and seed an AiModelPrice row for every deployment the product uses. Until the price table has a row, every run is refused: nothing could be costed, so the cap would be blind.

Seed prices in a journaled script of the product's own, after the table script. Prices are millionths of a US dollar per million tokens, taken from the provider's price list for that deployment's model; a price change is a NEW row with a later EffectiveFromDateTimeUtc, never an update, so past usage keeps the price it was charged at. The Deployment value is the deployment name exactly as Ai:Deployments maps it (and jev:{model} for Jev):

-- Scripts/20260927_1200_SeedAiModelPrice.sql (numbers are placeholders: use the provider's current list)
IF NOT EXISTS (SELECT 1 FROM dbo.AiModelPrice WHERE Deployment = N'gpt-4o-mini')
    INSERT INTO dbo.AiModelPrice (Deployment, InputMicroUsdPerMillionTokens, OutputMicroUsdPerMillionTokens, EffectiveFromDateTimeUtc)
    VALUES (N'gpt-4o-mini', 150000, 600000, '2026-09-27T00:00:00+00:00');

One row per deployment per environment database: a deployment present in production but unpriced there is charged the highest price on record (and logged), and an empty table refuses every run.

Upgrading a database that already has the tables. A column added after a table first shipped comes in the table script as a guarded ALTER, beside the CREATE, so re-running the package's current script brings an existing table up to shape. A product whose AiUsageEvent table was created from 0.2.0's script owes one journaled migration of its own for IsAbandoned (existing rows read as completed runs); copy this batch, which is the one in AiUsageEvent.sql, and update the product's own Tables/AiUsageEvent.sql to the new CREATE:

IF OBJECT_ID(N'dbo.AiUsageEvent', N'U') IS NOT NULL
   AND COL_LENGTH(N'dbo.AiUsageEvent', N'IsAbandoned') IS NULL
BEGIN
    ALTER TABLE [dbo].[AiUsageEvent] ADD [IsAbandoned] BIT NOT NULL CONSTRAINT [DF_AiUsageEvent_IsAbandoned] DEFAULT (0);
END
GO

A product whose AiUsageEvent table was created from 0.7.2's script or earlier owes one for CachedInputTokens (existing rows recorded none), the next batch in AiUsageEvent.sql:

IF OBJECT_ID(N'dbo.AiUsageEvent', N'U') IS NOT NULL
   AND COL_LENGTH(N'dbo.AiUsageEvent', N'CachedInputTokens') IS NULL
BEGIN
    ALTER TABLE [dbo].[AiUsageEvent] ADD [CachedInputTokens] BIGINT NOT NULL CONSTRAINT [DF_AiUsageEvent_CachedInputTokens] DEFAULT (0);
END
GO

Every product copies the two spend-cap indexes into a journaled migration of its own (the batches in AiUsageEvent.sql; without them each capped run scans today's rows): the tenant cap's below, and IX_AiUsageEvent_Surface_OccurredDateTimeUtc on ([Surface], [OccurredDateTimeUtc]) INCLUDE ([CostMicroUsd]) for the surface cap, guarded the same way.

IF OBJECT_ID(N'dbo.AiUsageEvent', N'U') IS NOT NULL
   AND NOT EXISTS (SELECT 1 FROM sys.indexes
                   WHERE object_id = OBJECT_ID(N'dbo.AiUsageEvent')
                     AND name = N'IX_AiUsageEvent_ScopeKey_OccurredDateTimeUtc')
BEGIN
    CREATE NONCLUSTERED INDEX [IX_AiUsageEvent_ScopeKey_OccurredDateTimeUtc]
        ON [dbo].[AiUsageEvent] ([ScopeKey] ASC, [OccurredDateTimeUtc] ASC)
        INCLUDE ([CostMicroUsd]);
END
GO

A product whose AiModelPrice table was created from 0.7.2's script or earlier owes one to accept a zero output price, which an embedding deployment's row needs (see Embeddings): these two batches from AiModelPrice.sql replace CK_AiModelPrice_Positive with CK_AiModelPrice_Prices (input above 0, output 0 or more); update the product's own Tables/AiModelPrice.sql to the new CREATE.

IF OBJECT_ID(N'dbo.CK_AiModelPrice_Positive', N'C') IS NOT NULL
BEGIN
    ALTER TABLE [dbo].[AiModelPrice] DROP CONSTRAINT [CK_AiModelPrice_Positive];
END
GO

IF OBJECT_ID(N'dbo.AiModelPrice', N'U') IS NOT NULL
   AND OBJECT_ID(N'dbo.CK_AiModelPrice_Prices', N'C') IS NULL
BEGIN
    ALTER TABLE [dbo].[AiModelPrice] ADD CONSTRAINT [CK_AiModelPrice_Prices]
        CHECK ([InputMicroUsdPerMillionTokens] > 0 AND [OutputMicroUsdPerMillionTokens] >= 0);
END
GO

A product whose AiToolCall table was created from 0.2.1's script or earlier owes one for ResultText (existing rows recorded no reply); copy this batch, which is the one in AiToolCall.sql, and update the product's own Tables/AiToolCall.sql to the new CREATE:

IF OBJECT_ID(N'dbo.AiToolCall', N'U') IS NOT NULL
   AND COL_LENGTH(N'dbo.AiToolCall', N'ResultText') IS NULL
BEGIN
    ALTER TABLE [dbo].[AiToolCall] ADD [ResultText] NVARCHAR(MAX) NULL;
END
GO

AiAgentSession is a table ApplyWeborityAi() maps from the release that added tool approvals, so every product on the core copies AiAgentSession.sql whole (its migration batch is the whole script) and adds it to its own Tables/, even before it uses an approval: a schema check compares the mapping against the database.

Configuration

{
  "Ai": {
    "Enabled": true,
    "AzureOpenAi": { "Endpoint": "https://<resource>.openai.azure.com" },
    "Deployments": { "chat": "gpt-4o-mini", "reasoning": "gpt-5-mini" },
    "DefaultDeployment": "chat",
    "DailySpendCapMicroUsd": 25000000,
    "SpendCaps": [
      { "Surface": "invoice-assistant", "DailySpendCapMicroUsd": 5000000 },
      { "Surface": "judgment.ticket-routing", "DailySpendCapMicroUsd": 1000000 }
    ],
    "ScopeDailySpendCapMicroUsd": 2000000,
    "MaxToolIterations": 10,
    "ReasoningEffort": "Medium",
    "EnableSensitiveData": false,
    "RecordToolResults": false,
    "ToolResultMaxLength": 8000,
    "ToolResultRetentionDays": 90,
    "AgentSessionExpiresAfterMinutes": 1440,
    "ManagedIdentityClientId": null
  }
}

On Foundry, "Foundry": { "ProjectEndpoint": "https://<resource>.services.ai.azure.com/api/projects/<project>" } takes the place of AzureOpenAi. Only the registered transport's section is read.

• Enabled switches AI off for the whole host, explicitly. It is true unless set. With "Enabled": false the host needs no other Ai key, no transport section and no context factory, even if Startup still calls AddWeborityAi and the transport registration; IAiAvailability.IsEnabled is false, and asking IAiAgentFactory for an agent throws AiDisabledException, as does a Jev judgment on a host with the core registered. Check IAiAvailability to hide a feature rather than catching the exception. A missing section never switches AI off: left on, it stops the host as below. Only an Ai:ApiKey still stops a switched-off host.
• Switched on, everything is validated at startup: no transport, the transport's endpoint missing or not https, no deployments, a DefaultDeployment that is not a key of Deployments, a cap that is not positive, a SpendCaps entry with no Surface, a cap below 1, a surface named twice (ignoring case) or holding :, a ManagedIdentityClientId that is not a GUID, a ReasoningEffort that is not one of None, Low, Medium, High, ExtraHigh, or no IDbContextFactory<TContext> or IHostEnvironment stops the host.
• Code names a deployment key, configuration names the deployment. An unknown key throws when the agent is built; there is no fallback.
• DailySpendCapMicroUsd is millionths of a US dollar per UTC day (25000000 is 25 USD).
• A cap is checked before a run, not reserved, so runs that start together can each pass just under it: the day's spend can overshoot a cap by about the number of concurrent runs times one run's cost. Set caps with that margin in mind.
• Each instance holds today's spend for five seconds. The gate reads the day's sums from the database at most once every five seconds per process, and adds each run this process records at once, so one instance alone stops at its cap. Spend by other instances can go unseen for up to five seconds, so the overshoot also grows by about five seconds of spend for every other instance running. A refusal is always checked against a fresh read.
• SpendCaps caps one surface at a time, on top of the overall cap. Surface is matched exactly against the usage rows' surface: an agent's Name, or judgment.{use} for a judgment use on either provider. A run first checks today's spend of its own surface against that surface's cap, then the day's total against DailySpendCapMicroUsd. A surface with no entry is bound by the overall cap alone. A surface never contains :, the configuration path separator: an agent name with one is refused when the agent is built, and a SpendCaps surface with one stops the host.
• Each tenant can have its own daily cap, so one tenant cannot use up the host's budget and switch AI off for the rest. After the surface cap, a run whose actor has a ScopeKey checks that tenant's spend today against its cap, then the overall one. The cap comes from the product's IAiScopeSpendCaps, registered scoped, when there is one (so a cap can follow the tenant's plan; 0 refuses the tenant), else from ScopeDailySpendCapMicroUsd (at least 1). With neither, tenants have no cap of their own. Work with no tenant is never tenant-capped. A refused run throws AiSpendCapExceededException with ScopeKey set.
• Sign-in is chosen from the host environment, never discovered. In the Local or Development environment it is the developer's az login (AzureCliCredential, pinned to AZURE_TENANT_ID when that is set); in every other environment it is the managed identity (ManagedIdentityCredential), system-assigned unless Ai:ManagedIdentityClientId names a user-assigned one by its client id. The host must register IHostEnvironment (Host.CreateDefaultBuilder does), or it stops at startup. DefaultAzureCredential is not used: on the Foundry pipeline's token path it fails on an unreachable managed-identity endpoint instead of falling through to az login. Azure OpenAI asks for a token for https://cognitiveservices.azure.com/.default on each request (on a developer machine the az login token is held until shortly before it expires, so the CLI is not run per request), so the identity needs the Cognitive Services OpenAI User role on the resource.
• Neither transport has an API key path, so an Ai:ApiKey setting stops the host rather than reading as configured.
• The package subscribes the host's OpenTelemetry tracer and meter providers to its sources; a host that exports through its own pipeline needs nothing more.

Building and running an agent

_actor.Set("user:" + user.PublicId, account.PublicId);   // once per scope, at the entry point

var agent = _agents.CreateAgent(new AiAgentSpec
{
    Name = "invoice-assistant",          // also the surface its usage is recorded under
    Instructions = InvoicePrompts.System,
    DeploymentKey = "chat",              // optional: Ai:DefaultDeployment otherwise
    Tools = [_invoiceTools],             // objects with [AgentFunction] methods
    ResponseType = typeof(InvoiceAnswer) // optional: the reply must match this record's schema
});

var response = await agent.RunAsync(question, cancellationToken: cancellationToken);
var answer = response.ReadResult<InvoiceAnswer>();

• The actor is required. AiActorContext is scoped; set it once per request or job with an opaque actor reference and the tenant key (null for work that belongs to no tenant). A run with no actor is refused before the model call. Tools read ScopeKey server-side to scope their queries; the model never sees it.
• Each run has an id. It is taken when the run starts (past the spend cap and the actor check) and stamped as RunId on the run's AiUsageEvent row and every AiToolCall row it made. Read it after the run from the scoped AiRunContext (_runs.RunId) to tie your own record to those rows; a Jev judgment on a host with the core registered gets one too. Like the actor, it assumes one run at a time per scope. A retried run keeps one id across its attempts, so a failed attempt's tool calls carry the same id as the usage row the successful attempt writes.
• Reserve the id before the run when your own record must exist first (a quota row, a job record): var runId = _runs.ReserveRun();, write your record under it, then run. The next run begun in the scope (a streamed run once its stream is first read) carries that id on its usage and tool-call rows, across retries. The run consumes the reservation whether it reaches the model or is refused by the spend cap or the actor check, and only a run that reached the model takes the id: after the run, _runs.HasRun && _runs.RunId == runId is true exactly when the model was called; otherwise nothing was sent or spent and you release your record. A second ReserveRun() before a run has consumed the first throws, as a second AiActorContext.Set does, because one id handed out twice would tie two of your records to one run; call ReleaseReservation() when you reserved and then decided not to run. Reserving while a run is open throws. A Jev judgment consumes a reservation the same way.
• AiSpendCapExceededException is thrown before the model call once today's cost reaches a cap. Its Surface names the surface whose own cap was reached, or is null when the overall cap was. Nothing was sent or spent.
• ReadResult<T> reads the final assistant message only. A ResponseType must serialize as a JSON object: wrap a list or a number in a record.
• A ResponseType is sent in OpenAI strict structured-output mode, so on the Azure OpenAI and Foundry transports the model's reply always matches the schema (the Anthropic transport ignores the strict setting). Strict mode closes every object to the properties its schema names, and returns an open-key member empty without an error. So a ResponseType with such a member, at any depth, throws ArgumentException at CreateAgent, as a non-object type does, naming the type and the member: a dictionary, JsonElement, JsonObject, object, or anything else whose schema allows unnamed keys. Model it as a record with named properties, or a list of key and value records. Strict covers the reply only: every tool's schema stays non-strict.
• A schema you wrote yourself goes in ResponseSchema (a JsonElement whose root is "type": "object"), with ResponseSchemaName (required: 1 to 64 letters, digits, _ or -) and an optional ResponseSchemaDescription. It is sent in OpenAI strict structured-output mode (Microsoft.Extensions.AI's "strict" option; each tool sets its own off, so tool schemas stay non-strict). The Azure OpenAI and Foundry transports enforce it; the Anthropic transport ignores the strict setting, so there the reply is not guaranteed to match the schema. Set ResponseType or ResponseSchema, never both; setting both, a schema whose root is not an object, a missing or malformed name, or a name without a schema throws at CreateAgent. Read the reply into your own type with ReadResult<T> ([JsonPropertyName] maps snake_case names). The OpenAI adapter rewrites a schema to OpenAI's strict subset before sending it: it adds "additionalProperties": false and marks every property required where the schema does not, and moves keywords it treats as unsupported (minimum, maximum, pattern, format, minLength, maxLength, minItems, maxItems, default and a few more) into the node's description, where the model reads them as guidance rather than a constraint. A schema already in that subset goes out unchanged.

using var schema = JsonDocument.Parse(ExtractionSchemas.Invoice);
var agent = _agents.CreateAgent(new AiAgentSpec
{
    Name = "invoice-extraction",
    DeploymentKey = "extraction",        // a different Ai:Deployments entry from the chat's
    ReasoningEffort = ReasoningEffort.Low, // overrides Ai:ReasoningEffort for this agent only
    ResponseSchema = schema.RootElement, // copied, so the document may be disposed after CreateAgent
    ResponseSchemaName = "invoice_extraction"
});
var invoice = (await agent.RunAsync(prompt, cancellationToken: cancellationToken)).ReadResult<ExtractedInvoice>();

• Temperature is dropped, with a warning, for reasoning deployments (gpt-5 and later, the o-series), which reject it.
• ReasoningEffort (Microsoft.Extensions.AI's ReasoningEffort) is the mirror image: a spec's value wins over Ai:ReasoningEffort, and the result is sent only to reasoning deployments, on either transport. The configured default is skipped quietly for a deployment that does not reason; a spec that asks for one there is dropped with a warning. Unset on both, no reasoning option is sent and the model uses its own default.
• MaxOutputTokens caps one reply's length (at least 1), sent on both transports; on the Azure OpenAI wire it is max_completion_tokens, which reasoning deployments accept and which counts their reasoning too. Unset, the deployment's own limit applies.

A plain completion

A one-shot completion is an agent with no tools: it is gated, metered and retried like any run, and needs no API of its own.

var summary = (await _agents.CreateAgent(new AiAgentSpec { Name = "ticket-summary", Instructions = "Summarise in one line.", MaxOutputTokens = 200 })
    .RunAsync(ticket.Body, cancellationToken: cancellationToken)).FinalReplyText();

Embeddings

var generator = _embeddings.CreateGenerator(new AiEmbeddingSpec
{
    Name = "ticket-search",          // the surface its usage is recorded and capped under
    DeploymentKey = "embedding",     // required: an Ai:Deployments key naming an embedding deployment
    Dimensions = 1536                // optional: for a model that can shorten its vectors
});

var embeddings = await generator.GenerateAsync(chunks, cancellationToken: cancellationToken);

• IAiEmbeddingFactory (scoped) returns Microsoft.Extensions.AI's IEmbeddingGenerator<string, Embedding<float>>, answered by whichever transport the host registered. Each call carries an agent run's guarantees: it is refused before anything is sent once a cap is reached or when no actor is set, a transient failure is retried with the same bounds, and it writes one AiUsageEvent with operation Embed and its own run id.
• Price an embedding deployment with an output price of 0: it writes no output tokens. The price check allows 0 for output (never for input); a product on an older table runs the batch under Upgrading a database that already has the tables.

Tools

public sealed class InvoiceTools
{
    [AgentFunction("get_invoice")]
    [Description("Looks up one invoice.")]
    public Task<InvoiceSummary> GetInvoiceAsync(
        [LoggableArgument, Description("The invoice id.")] long invoiceId,
        [Description("What the user asked.")] string question,
        CancellationToken cancellationToken) => ...;
}

• Public [AgentFunction] methods, instance or static, become tools; the plugin name recorded is the object's type name. An object with no such method, or two tools with one name, throws when the agent is built.
• Each call writes an AiToolCall row: agent, tool, plugin, arguments, outcome, duration, actor, scope and run id. Argument values are redacted unless the parameter carries [LoggableArgument]; the names are always recorded.
• A tool that throws UnauthorizedAccessException is recorded Denied and the model is told "Access denied."; the reason stays in the log. Any other exception is recorded Failed and the model is told the tool failed.
• A run makes at most Ai:MaxToolIterations tool round trips, or the spec's own MaxToolIterations (1 to 100) when it sets one; hitting the cap logs a warning naming the agent.
• What a tool returned is recorded only when Ai:RecordToolResults is true, and it is false unless set. A tool's reply can carry customer content that the argument redaction otherwise keeps out of the audit, so turn it on deliberately. Switched on, a call that succeeded stores its reply in AiToolCall.ResultText: a string as returned, anything else as JSON. A reply longer than Ai:ToolResultMaxLength characters (default 8000, 256 to 1,000,000) is cut to that many and ends with a marker giving its original length. A denied or failed call stores nothing, because exception text can carry internals. The reply never reaches the log.
• A tool the person confirms before it runs is marked [AgentFunction("raise_ticket", RequiresApproval = true)]. It becomes the agent framework's ApprovalRequiredAIFunction, and its agent is wrapped, outermost, in the framework's ToolApprovalAgent, which hands the caller one request at a time. The run stops with a ToolApprovalRequestContent instead of calling the tool; the call runs (and is audited) only when the caller sends back the request's CreateResponse(true) on the same agent session, and a rejected call never runs. Between two requests the session is held by AiAgentSessionStore (scoped), the framework's AgentSessionStore over the product's database (its base is experimental, so Webority.Ai alone suppresses MAAI001, a recorded exception; the string-keyed methods below need no experimental type): SaveAsync(agent, key, session), GetAsync(agent, key) (null when none, expired, or another actor's), DeleteAsync, and DeleteExpiredAsync() from the product's own timer; a saved session expires after Ai:AgentSessionExpiresAfterMinutes (default 1440, 1 to 43200). The chat endpoint does all of this for a chat turn (see Chat). This is for the same person, now; an action another person approves later belongs in Webority.Ai.Actions.
• Recorded replies expire after Ai:ToolResultRetentionDays (default 90, 1 to 3650), but only when the product clears them: the package runs no timer. Call AiToolCallRetention.ClearExpiredResultsAsync(cancellationToken) from a daily timer of the product's own (a Functions timer or a hosted job, inside a DI scope, since it is scoped). It sets ResultText to null on older rows, keeps the rest of each row, and returns how many it cleared. It runs whether or not recording is on, so replies recorded before it was switched off still expire. A host that only clears replies (a Functions app, say) can register the core with Ai:Enabled false: no transport or agent setting is then needed, but the host must still register IDbContextFactory<TContext>, because that check is switched off with the rest and the clear opens its context through it.

Cost

Every run writes one AiUsageEvent: surface (the agent name), operation (Run or RunStreaming, or the spec's Operation when set, as a judgment sets its operation name), deployment, input and output tokens, CostMicroUsd, actor, scope, run id and time. CachedInputTokens is the part of the input tokens the provider read from its prompt cache: it is counted inside InputTokens, as the provider reports it, and charged at the input price, so caching never lowers the recorded cost. A streamed run is metered however it ends. Read to the end, it writes its row as above. Abandoned after at least one update (the caller breaks out of the loop, its token is cancelled, or the stream throws), it still writes one row, with IsAbandoned true and the tokens the model had reported by then, so the spend counts against the caps. Those tokens are a floor: the provider reports a model call's usage only when the call finishes, so a call cut off mid-answer adds nothing, and only the finished model calls before it (tool rounds) are charged. A stream that ends before its first update writes no row: the model reported nothing, and a failed attempt there is what the retry makes again. Prompt tokens the provider may bill for a request cancelled in flight are therefore not counted. Cost is integer micro-USD, rounded up, from the AiModelPrice row in effect at the time. A deployment with no price is charged the highest input and output price on record and logged, never zero. A usage or tool-call row that cannot be written is counted on webority.ai.write_failures and logged; it never fails the run.

Spend alerts

A person hears about a cap before it refuses runs: once today's spend reaches Ai:SpendAlert:Percent (default 80, 1 to 100) of the overall cap or of an Ai:SpendCaps surface's cap, the product's notifier is called, once per cap per UTC day.

// Startup.ConfigureServices, beside AddWeborityAi
services.AddWeborityAiSpendAlerts<AppDbContext>(_configuration);
services.AddScoped<IAiSpendAlertNotifier, OpsEmailSpendAlertNotifier>();   // the product's own channel

// AppDbContext.OnModelCreating, beside ApplyWeborityAi()
modelBuilder.ApplyWeborityAiSpendAlerts();

// The product's own timer, every 15 minutes, inside a DI scope
var check = await scope.ServiceProvider.GetRequiredService<IAiSpendAlerts>().CheckAsync(DateTimeOffset.UtcNow, cancellationToken);

Copy content/schema/AiSpendAlert.sql into the product's Scripts/ (idempotent, guarded; the migration batch is the whole script).

• The caps are the spend gate's own (Ai:DailySpendCapMicroUsd, Ai:SpendCaps), summed the way the gate sums them: today's usage rows from UTC midnight, abandoned runs included. A surface's cap and the overall cap alert independently. Tenant caps raise no alert.
• Once per cap per day, across overlapping checks. Each alert is claimed as an AiSpendAlert row (CapKey is all or surface:{name}, AlertDate the UTC day) before the notifier is called; a unique index on the pair decides two checks racing. A new UTC day alerts again.
• A notifier that throws gives the alert back: the claim is deleted, the exception is rethrown from CheckAsync, and the next check tries again. Caps after it in that check wait for the next one.
• AiSpendAlertNotice carries the Surface (null for the overall cap), the cap and today's spend in micro-USD, the Percent of the cap spent (rounded down; it can pass 100) and the AlertDate. CheckAsync returns how many caps it checked, how many were over the threshold and how many it notified.
• The package runs no timer. With Ai:Enabled false nothing is checked and no notifier is needed; switched on, startup stops without AddWeborityAi, an IAiSpendAlertNotifier or an IDbContextFactory<TContext>.

Testing a product's agents

var model = new FakeChatClient()
    .CallTool("get_invoice", new Dictionary<string, object> { ["invoiceId"] = 42L, ["question"] = "total?" })
    .ReplyJson(new InvoiceAnswer("12,500"), inputTokens: 900, outputTokens: 40);

services.AddScoped<IAiAgentFactory>(_ => new FakeAiAgentFactory(model, deployment: "gpt-4o-mini"));

The fake builds real agent-framework agents with the package's own tool discovery, reply schema, temperature and reasoning rules, so tools run for real; pass the host's Ai:ReasoningEffort as reasoningEffort to have it applied as the transports apply it, and a loggerFactory to assert what the package logs, such as the warning when a spec's temperature or reasoning effort is dropped. It does not retry, gate spend, write usage or wrap an agent with an approval-required tool in ToolApprovalAgent; those are the package's, tested here, so test an approval flow over the real chain. A model call with no scripted reply left throws. A test host that keeps the product's AddWeborityAi and transport registration still needs the transport's endpoint in its configuration, or startup validation stops it.

Over the real chain

To test an agent the way it runs in production, keep the product's own registration and replace only Azure OpenAI's endpoint:

var server = new FakeAzureOpenAiServer()
    .CallTool("get_invoice", new Dictionary<string, object> { ["invoiceId"] = 42L, ["question"] = "total?" })
    .ReplyJson(new InvoiceAnswer("12,500"), inputTokens: 900, outputTokens: 40);

services.AddWeborityAi<AppDbContext>(configuration);
services.AddWeborityAiAzureOpenAi(configuration);
services.UseFakeAzureOpenAi(server);   // after the transport registration, which it replaces

// ... run the product's code, then:
Assert.Equal(2, server.Requests.Count);   // each with its Uri, Authorization header and JSON body
Assert.Contains("get_invoice", server.Requests[1].Body);   // the tool call and its result, sent back to the model

Every request goes through the real OpenAI SDK, the Azure OpenAI transport, the retry, the spend gate, the instrumentation and the tool audit, so the usage and tool-call rows land in the product's database as they would in production. A reply's cachedInputTokens reports that part of its input tokens as read from the prompt cache. Fail(HttpStatusCode) answers the next request with an error status and ThrottleEveryRequest(retryAfterSeconds) answers every one with 429, to exercise the retry. A request with no scripted reply left fails. It streams: a streamed run (RunStreamingAsync) is answered as the service answers one, with Server-Sent Events carrying the reply text and a tool call's arguments in several deltas, the finish reason, and the usage chunk the SDK asks for (stream_options.include_usage), so the same script runs a streamed agent over the whole chain and writes the same usage and tool-call rows as a non-streamed run. Breaking out of a streamed run over it records the abandoned row described under Cost. The host still needs Ai:AzureOpenAi:Endpoint (FakeAzureOpenAiServer.Endpoint will do), an IHostEnvironment, a context factory and a seeded AiModelPrice row, as in production. The fake speaks Azure OpenAI's chat completions API only, so UseFakeAzureOpenAi refuses a host on the Foundry transport.

Chat

A product's assistant chat, on the core: conversations live in the product's database and belong to the actor that started them, so the model's history never comes from the browser (a forged earlier reply cannot reach it).

services.AddWeborityAi<AppDbContext>(_configuration);
services.AddWeborityAiAzureOpenAi(_configuration);
services.AddWeborityAiChat<AppDbContext>(_configuration);   // AiChatStore
services.AddWeborityAiChatEndpoint();                         // AiChatEndpoint, in Webority.Ai.Chat.AspNetCore

// AppDbContext.OnModelCreating
modelBuilder.ApplyWeborityAi();
modelBuilder.ApplyWeborityAiChat();

Copy content/schema/AiConversation.sql then AiConversationMessage.sql into the product's Scripts/. Ai:Chat is optional: HistoryMessages (10), MaxMessageLength (4000), MaxApprovalsPerTurn (5; past it the turn ends assistant-unavailable), AllowanceTimeZone ("UTC"; "Asia/Kolkata" resets the allowance at Indian midnight), RetentionDays (180), Retention (none).

A turn, from a controller (the actor is set first, as for any run):

actor.Set($"customer-user:{userPublicId}", customerPublicId.ToString());
await endpoint.WriteTurnAsync(HttpContext, new AiChatTurnRequest
{
    Surface = "assistant-chat",
    ConversationId = request.ConversationId,
    Message = request.Message,
    DailyLimit = 100,
    Prepare = turn =>
    {
        var tools = new object[] { new MyTools(currentUser), supportTools.Create(turn.ConversationPublicId.ToString("N")) };
        return new AiChatRun
        {
            Agent = agents.CreateAgent(new AiAgentSpec { Name = "assistant-chat", Instructions = Prompt, Tools = tools }),
            Guard = new AiReplyGuard(tools, Refusal, forbiddenTerms: ["FollowUp"]),
            Context = [new ChatMessage(ChatRole.User, "Context from the screen, not a question: ...")],
            Actions = () => support.PreparedActions.Cast<object>().ToList()
        };
    }
});
return new EmptyResult();

• Early failures answer as JSON { code, message }: 400 message-required / message-too-long, 404 conversation-not-found, 429 daily-limit (with resetsAtUtc), 400 message-not-allowed when the model provider's content filter refuses the prompt (a jailbreak attempt, say), 503 assistant-at-capacity / assistant-disabled / assistant-unavailable. Nothing is counted for any of them. AiChatTurnRequest.Copy overrides the words.
• Otherwise the response is text/event-stream: delta { text }, reset {} (discard what was shown: the model turned to a tool, or the guard will replace the reply), actions { actions } (cards to confirm, only for a stored reply), then one done { conversationId, messageId, reply, messagesLeftToday } whose reply is the reply of record, approval { conversationId, requestId, toolName, arguments } when the run paused for the person to confirm a tool call, or error { code, message }.
• A tool approval pauses the turn. When the agent (from IAiAgentFactory) calls a RequiresApproval tool, the stream ends with approval: the client shows its own card for toolName and arguments, and the question stays stored and counted but unanswered. The client answers on the same route with ConversationId and Approval = new AiChatApproval { RequestId, Approved } and no message; Prepare must build the same agent. The turn resumes from its saved session (AiAgentSessionStore, keyed by the conversation), the approved call runs or the rejected one is declined, and the reply streams to done as for a question, counting nothing more. A wrong request id, a conversation with no paused turn, a pause past Ai:AgentSessionExpiresAfterMinutes, an approval for a question the conversation has moved past, or a second answer to one request answers 404 approval-not-found (AiChatCopy.ApprovalNotFound) and changes nothing; an expired pause keeps its question counted. Only the request that claims the pause runs the call, and a resumed turn that fails after its claim is stored as answered with the words the person read, so asking again is a new turn and never a silent repeat. A new question in the conversation abandons the pause. Call AiAgentSessionStore.DeleteExpiredAsync() from the product's daily timer.
• The allowance is atomic: the question is stored and counted in one serializable transaction before the model is called, so concurrent requests cannot overshoot; on SQL Server a racing pair may deadlock and the product's EnableRetryOnFailure strategy runs the loser again. A failed, refused or abandoned turn (the person closed the tab) gives the question back.
• The guard is built from the tool objects the agent is given that turn, so a support package's tools are covered with the product's own. Forbidden terms match whole words, case-sensitively: pick words prose never uses.
• A visitor's conversation can follow them into their account: ClaimVisitorConversationAsync(surface, id, actor, scope, maxIdle) hands a tenantless conversation used within maxIdle to the person who just signed up from it; an owned or stale one is left alone.
• Erasure and retention: DeleteScopeAsync(scopeKey) for a tenant, DeleteActorAsync(actorReference) for a person, DeleteExpiredAsync() from the product's own daily timer.
• Retention per surface: Ai:Chat:Retention lists rules by exact Surface, each with either RetentionDays (1 to 3650) or KeepForever: true. DeleteExpiredAsync applies each listed surface's rule and RetentionDays to every other surface, so with no rules nothing changes. A rule with neither or both, a surface named twice (ignoring case) or holding : stops startup.

"Ai": { "Chat": { "Retention": [
  { "Surface": "website-chat", "RetentionDays": 180 },
  { "Surface": "assistant-chat", "KeepForever": true }
] } }

• The read endpoints (allowance, list, get, delete, rate) are over AiChatStore; MapAiChat maps them (below), or a product writes its own thin controllers.

Mapping the routes. MapAiChat maps what @webority/chat-react calls, so a product writes no controller:

var chat = endpoints.MapAiChat("/api/assistant", new AiChatMapOptions
{
    Surface = "assistant-chat",
    DailyLimitAsync = async (http, ct) => await plans.QuestionsPerDayAsync(http.User, ct),   // null for no limit
    Prepare = (context, turn) =>
    {
        var tools = new object[] { new MyTools(context.Http.User) };
        return new AiChatRun
        {
            Agent = agents.CreateAgent(new AiAgentSpec { Name = "assistant-chat", Instructions = Prompt, Tools = tools }),
            Guard = new AiReplyGuard(tools, Refusal),
            Context = [new ChatMessage(ChatRole.User, $"Context from the screen, not a question: {context.PageContext}")]
        };
    }
});
chat.RequireAuthorization();
chat.AddEndpointFilter(async (ctx, next) =>
{
    var user = ctx.HttpContext.User;
    ctx.HttpContext.RequestServices.GetRequiredService<AiActorContext>()
        .Set($"customer-user:{user.FindFirstValue("sub")}", user.FindFirstValue("customer"));
    return await next(ctx);
});

• Routes under the prefix: POST chat (body { message, conversationId?, pageContext?, approval? }, streamed as above), GET allowance ({ messagesLeftToday }), GET conversations?skip&take (newest first, take clamped to 1 to 100), GET conversations/{id}, DELETE conversations/{id} (204), PUT conversations/{id}/messages/{messageId}/rating (body { rating: "Up" | "Down", note? }, 204). Bodies are read and answers written with the contract's own JSON options, never the host's.
• The host owns authentication. MapAiChat returns the RouteGroupBuilder; add RequireAuthorization, rate limiting and the endpoint filter that sets AiActorContext, which applies to every route in the group. An actor never set fails the request loudly.
• Failures are { code, message }: the codes above, plus 400 request-invalid (an unreadable body or rating), 404 conversation-not-found and 404 message-not-found. AiChatMapOptions.Copy overrides the words.
• AiChatMapOptions.Prepare gets an AiChatTurnContext (Http, Surface, ConversationId, Message, IsApproval, DailyLimit, PageContext, the client's untouched pageContext JSON) beside the turn.

A gate can veto a turn. IAiChatTurnGate.CheckAsync(AiChatTurnContext, ct) returns AiChatGateResult.Allow or AiChatGateResult.Refuse(code, message, status = 403, resetsDateTimeUtc = null). Register any number with services.AddWeborityAiChatTurnGate<MyGate>() (scoped); they run in registration order, after the request is validated and before the question is stored or counted, and the first refusal wins. A refused turn writes nothing, counts nothing, never calls Prepare, and answers the refusal's { code, message } (and resetsAtUtc when given) as plain JSON, so write the message for the person. Gates run for an approval answer too (IsApproval); a gate that throws fails the request before anything is stored. It applies to WriteTurnAsync however it is reached, so a hand-written controller gets it as well. Use Prepare for building the agent and a gate for a decision, and DailyLimit for a number of questions.

Actions

Actions an agent proposes for another person to approve later: the tool records a proposal and returns at once, an authorised person approves or rejects it from a queue, and the one executor registered for its kind runs it. The capability lives in the executor, never in the tool. For the same person confirming a call in the chat, use [AgentFunction(RequiresApproval = true)] instead (see Tools).

services.AddWeborityAi<AppDbContext>(_configuration);
services.AddWeborityAiActions<AppDbContext>(_configuration);                         // IAiActions
services.AddWeborityAiActionExecutor<RefundExecutor, RefundRequest>("refund");       // one executor per kind
services.AddWeborityAiActionEndpoint();                                              // AiActionEndpoint, in Webority.Ai.Actions.AspNetCore

// AppDbContext.OnModelCreating
modelBuilder.ApplyWeborityAiActions();

"Ai": {
  "Actions": {
    "Kinds": [
      { "Kind": "refund", "Mode": "Approve", "ExpiresAfterMinutes": 1440 },
      { "Kind": "raise-ticket", "Mode": "Confirm", "ExpiresAfterMinutes": 30 },
      { "Kind": "tag-customer", "Mode": "Auto", "ExpiresAfterMinutes": 60 }
    ]
  }
}

Copy content/schema/AiAction.sql into the product's Scripts/; it creates both tables and their indexes and is idempotent, so the migration batch is the whole script. Ai:Actions:MaxPayloadLength (default 16384, 1 to 1048576) caps a proposal's payload in characters of JSON.

• Proposing is a call from an [AgentFunction] tool: await actions.ProposeAsync("refund", new RefundRequest(invoiceId, amountPaise), "Refund invoice INV-42 in full.", ct) returns an AiActionProposal (PublicId, Mode, ExpiresDateTimeUtc) at once; tell the model it awaits approval, and hand the product's card to the chat through AiChatRun.Actions. The actor and tenant are the AiActorContext's: a proposal needs a scope key, and only an actor in the same scope sees or decides it. The payload may be the executor's payload type or a type derived from it, and is stored whole as JSON of the executor's type in AiAction.PayloadJson, because the executor needs it; a payload over MaxPayloadLength throws ArgumentException; the queue (ListPendingAsync, GetAsync) shows a copy with every value redacted unless its property or record parameter carries [LoggableArgument], as for tool arguments.
• Modes: Confirm is decided by the proposer only; Approve by anyone in the scope but the proposer (who may still withdraw it by rejecting); Auto runs inside ProposeAsync and is recorded the same way, for reversible kinds only. Who may approve at all is the product's route authorization.
• Deciding: ApproveAsync(id) and RejectAsync(id, reason) return an AiActionDecision whose Status is Executed, Failed, Rejected, NotFound, AlreadyDecided, Expired or NotAllowed. An action is decided once: the outcome table's unique decision index lets exactly one of two racing approvals land, and only that one runs the executor. The decision commits in a transaction verified by its own row, so a retry after a lost commit acknowledgement finds it landed.
• Executors implement IAiActionExecutor<TPayload>.ExecuteAsync(payload, execution, ct) and return AiActionResult(Succeeded, Message), the message a plain sentence for the person. AiActionExecution carries the action id, kind, scope key, proposer and approver from the server, never from the payload. An executor that throws, a timeout included, is logged and recorded Failed with no message. A claimed action runs to the end whatever the caller does: the executor gets no caller token, and an outcome that cannot be stored is logged while the caller still hears what ran.
• Every outcome is a row: AiActionOutcome is append-only (Approved, Rejected, Expired, then Executed or Failed), with who and when. ListPendingAsync(page, pageSize) is the tenant's queue; GetAsync(id) an action with its outcomes.
• Expiry: an action past ExpiresAfterMinutes cannot be approved; call ExpireAsync() from the product's own timer to record the rest as Expired, 500 at a time; an action it cannot record is logged and left for the next sweep.
• Startup stops when a kind has no executor or two, an executor's kind is not listed, a kind is listed twice or holds :, an expiry is outside 1 to 43200 minutes, or MaxPayloadLength is outside its range.
• The endpoint: AiActionEndpoint.HandleAsync(http, actionId, new AiActionEndpointRequest { Approved, Reason }) on the product's own route, after the product sets AiActorContext. 200 { outcome, message } for a decision that landed; otherwise { code, message }: 401 unauthenticated, 400 ai-action-decision-required (no approved in the body, so nothing is decided) or ai-action-reason-too-long, 403 ai-action-not-allowed, 404 ai-action-not-found, 409 ai-action-decided, 410 ai-action-expired, 503 ai-action-unavailable.
• Not in v1: chained approvals, standing approval policies, reviewers and revert checks.

Judgment

Calibrated judgments over text: IAiJudge answers eleven operations (verify, screen, compare, find, rerank, classify, decide, probability, extract, review, gate) with typed results and an AiJudgment<T> carrying Value, Confidence, Action (Review, Auto, Block), Usage and Model. Every call names a use, and the use chooses the provider and the thresholds.

• Webority.Ai.Judgment: the provider-neutral contract, and the one composition of each operation: jev-mcp 0.9.0's state, question wording, limits and fail-closed validation. A provider only answers the question set, so a use switches provider in configuration and the same caller code gets the same result shapes and actions.
• Webority.Ai.Judgment.Jev: the TypeSafe Jev provider, one POST https://api.typesafe.ai/v1/systemone per call.
• Webority.Ai.Judgment.Agent: the own-model provider, one agent run per call through IAiAgentFactory with a typed reply: a probability for every label of every question. It needs the core (AddWeborityAi and a transport) and answers on whichever transport the host registered, Azure OpenAI or Foundry.

An extraction whose patterns match nothing makes no call on either provider, and the model only ever chooses among the regex matches, so an extracted value is always verbatim document text.

Setup

services.AddWeborityAi<AppDbContext>(_configuration);   // required by the agent judge; Jev is capped and metered through it when present
services.AddWeborityAiAzureOpenAi(_configuration);      // the core's transport, or AddWeborityAiFoundry
services.AddWeborityAiJudgmentAgent();
services.AddWeborityAiJudgmentJev();                     // either provider, or both

{
  "Ai": {
    "Judgment": {
      "Jev": { "ApiKey": "<secret>", "Model": "jev-latest" },
      "Uses": {
        "ticket-routing": { "Provider": "Agent", "AutoAccept": 0.9, "MinimumMargin": 0.6 },
        "page-screen": { "Provider": "Jev", "BlockAt": 0.8, "ReviewAt": 0.3 }
      }
    }
  }
}

Everything is validated at startup: a missing Jev key, an unregistered provider, the agent judge registered without the core, a use name over 119 characters, a threshold outside 0 to 1, ReviewAt above AutoAccept or BlockAt, an EscalateTo that is not a use, names the same provider or escalates again, or a DeploymentKey on a provider other than Agent or not in Ai:Deployments stops the host. Every call names a use:

var judgment = await judge.ClassifyAsync("ticket-routing", request, cancellationToken);
if (judgment.Action == AiJudgmentAction.Auto) { /* act on judgment.Value */ }

Inject the unkeyed IAiJudge; it routes each call to the provider the use names. The agent judge is scoped, as the agent factory is, so resolve IAiJudge inside a request or job scope.

An unset threshold takes the operation's jev-mcp default: AutoAccept 0.8 (verify, decide, review, gate) or 0.85 (classify, compare, extract, probability); ReviewAt 0.25 for screen and min(0.5, AutoAccept) for review and gate; BlockAt 0.75; MinimumMargin 0.5; CompositeFloor 0.7. Both providers apply the same thresholds the same way.

Escalating uncertain answers

A use can hand every judgment it returns as Review to another use with EscalateTo, so the dearer model is paid only for the uncertain tail. The target is an ordinary use, with its own provider, thresholds and Ai:SpendCaps surface, and its judgment (value, action, usage and model) is the one returned. Both legs are metered, each under its own judgment.{use}. The agent judge runs on Ai:DefaultDeployment unless the use names another Ai:Deployments key in DeploymentKey.

"Uses": {
  "ticket-routing": { "Provider": "Jev", "AutoAccept": 0.9, "EscalateTo": "ticket-routing-agent" },
  "ticket-routing-agent": { "Provider": "Agent", "DeploymentKey": "reasoning", "AutoAccept": 0.8 }
}

The target must name a different provider and must not escalate again. The two providers' confidences are not on one scale, so tune the target's thresholds on their own. Each leg is its own run, so a reserved run id (AiRunContext.ReserveRun) is taken by the first leg.

Actions

Review is the zero value of AiJudgmentAction, and of the native AiReviewAction, AiExtractStatus and AiScreenRecommendation, so an unset action, status or recommendation fails closed to a person. Compare them by name, never by numeric value.

Confidence on the result is the weakest confidence the action rests on, null when any of it is unknown or the answers are yes/no probabilities. Jev reports how peaked its distribution is; the agent judge's is the probability the model gave the winning label (for a rubric, its most probable score). Neither is the probability the answer is correct: tune thresholds against observed outcomes, per provider.

A malformed answer never throws: a missing answer, a distribution that is not exactly the question's labels summing to 1, an agent judge reply naming a question that was not asked, or one that is not the typed shape comes back as InvalidResponse with a non-automatic action.

Metering

Both providers write an AiUsageEvent per call when the core is registered, under the surface judgment.{useName} and the operation name (AiJudgmentOperations: Classify, Review and so on), so one query reads a use's spend by operation whichever provider served it:

A Jev call that may have been charged without its answer arriving whole (a timed-out or cancelled attempt, or a success status with a body that is not JSON) still writes its row, flagged IsAbandoned with no tokens, as an abandoned agent stream does. A call TypeSafe refused writes none.

An unpriced deployment is charged the highest price on record and logged, as for any agent run. With the core registered, both providers refuse a call before it is sent when no actor is set in AiActorContext, and once the day's recorded spend is at or above the use's own Ai:SpendCaps entry for judgment.{use}, the tenant's cap, or Ai:DailySpendCapMicroUsd (AiSpendCapExceededException, from the same gate an agent run passes, so an empty AiModelPrice table refuses Jev calls too). Agent judge calls also retry.

Without the core, Jev runs, meters nothing and is never capped: the gate and recorder are looked up at call time, so a host that registers only AddWeborityAiJudgmentJev() needs no database.

Judgment ledger

The ledger is its own package, Webority.Ai.Judgment.Ledger, so Webority.Ai.Judgment stays provider-neutral and never references Webority.Ai. Call services.AddWeborityAiJudgmentLedger() beside AddWeborityAi<TContext>() (it calls AddWeborityAiJudgment() itself; the host stops at startup when the core is missing) and every call through the unkeyed IAiJudge writes one AiJudgment row per judgment leg, through a context of its own, so a person can review what the judge decided and the product can measure it against what people decide. An escalated judgment writes two rows: the first leg's, and the target's with EscalatedFromJudgmentId pointing at it. A failed ledger write is counted (AiMetrics) and logged and never fails the judgment. Without the package nothing is recorded and judgments work as before: the router calls an IAiJudgmentRecorder only when one is registered, and a keyed provider resolved directly bypasses the router and records nothing.

• Tie a judgment to your record with an ambient scope, not a parameter: using (AiJudgmentSubject.Begin(ticket.PublicId.ToString())) { await judge.ClassifyAsync(...); } stores the id on each row's SubjectRef (an opaque string, at most 128 characters). Scopes nest and the innermost wins.
• The review queue and outcomes are IAiJudgmentLedger, scoped, resolved with IAiJudge's scope. ListReviewQueueAsync(useName, page, pageSize) returns the use's Review judgments with no outcome yet, newest first, leaving out a leg that was escalated (the target's row stands in its place). GetAsync(publicId) returns one judgment with its outcomes. RecordOutcomeAsync(publicId, outcome, correctedValue) appends a decision as the AiActorContext actor: a Corrected outcome carries the right value as a JsonNode and the others carry none. A judgment can be decided again, and every decision stays. An actor with a scope key reads and decides only that tenant's rows. Results are AiJudgmentEntry records with the PublicId.
• Request text is opt-in. By default only InputHash is kept. A use with StoreInputText: true also keeps the request's JSON in InputText and must set InputTextRetentionDays (1 to 3650); InputTextRetentionDays on a use that does not store text is refused at startup. The package runs no timer: call IAiJudgmentLedger.PurgeExpiredInputTextAsync(cancellationToken) from the product's own daily job. It sets InputText to null on rows older than their use's retention, keeps every other column, and sweeps only uses still configured with StoreInputText.

"Uses": {
  "ticket-routing": { "Provider": "Jev", "StoreInputText": true, "InputTextRetentionDays": 30 }
}

Mapping and schema: call modelBuilder.ApplyWeborityAiJudgment(); beside ApplyWeborityAi() in OnModelCreating, and copy Schema/AiJudgment.sql from the package (content/schema/ in the nupkg) into the product's Scripts/. The script creates both tables and their indexes and is idempotent. Add the package, the mapping and the script in the same release: with the package registered and the tables missing, each judgment logs a ledger write failure and carries on. The migration batch is the whole script, named Scripts/YYYYMMDD_HHMM_AiJudgment.sql in the product.

Jev transport

JevSystemOneClient is a typed client. It retries only 408, 429 and 5xx (529 included), up to MaxAttempts (default 3), with 500 ms doubling backoff capped at 5 s and up to a quarter jitter, honoring retry-after-ms and Retry-After up to 60 s. A timed-out attempt (AttemptTimeout, default 10 s), a connection failure, any other status and a cancelled call are never retried. It follows no redirect: every request carries the Bearer key, so a 3xx is a non-retryable JevApiException. Error text has the key redacted before it is truncated; logged header values are all redacted.

Browser and mobile

Webority.Ai.Judgment.AspNetCore lets a browser or app reach extract and classify through the product's own route, so the product's authentication and rate limiting apply, exactly as for chat. The browser sends only text and field names; the catalogue, patterns, descriptions and use come from configuration, and the actor from the signed-in user.

services.AddWeborityAiJudgment();                 // with a provider package, as under Setup
services.AddWeborityAiJudgmentEndpoint();         // AiJudgmentEndpoint; binds and validates Ai:Judgment:Endpoint

app.MapPost("/api/ai/judgment/{use}", (HttpContext http, string use, AiJudgmentEndpointRequest body, AiJudgmentEndpoint ep) => ep.HandleAsync(http, use, body))
    .AddEndpointFilter(async (context, next) =>
    {
        var user = context.HttpContext.User;
        if (user.Identity?.IsAuthenticated == true)
            context.HttpContext.RequestServices.GetRequiredService<AiActorContext>()
                .Set($"customer-user:{user.FindFirstValue("sub")}", user.FindFirstValue("customer"));
        return await next(context);
    })
    .RequireAuthorization()
    .RequireRateLimiting("ai-judgment");

As with chat, the product sets AiActorContext before HandleAsync (here in an endpoint filter, with its own claim names); an authenticated request that reaches the handler with no actor set throws, so a missing filter fails loudly rather than running unattributed.

"Ai": { "Judgment": { "Endpoint": { "Uses": [
  { "Use": "invoice-fields", "Operation": "extract", "Fields": [
      { "Name": "invoice-number", "Pattern": "INV-\\d{6}", "Description": "The invoice number." },
      { "Name": "total", "Pattern": "[\\d,]+\\.\\d{2}", "Description": "The amount due." } ] },
  { "Use": "ticket-routing", "Operation": "classify", "Fields": [
      { "Name": "billing", "Description": "Questions about invoices or payments." },
      { "Name": "technical", "Description": "Something is broken or will not load." } ] }
] } } }

• The exposure list is Ai:Judgment:Endpoint:Uses. Only a listed use is reachable; every other use, configured or not, answers 404 ai-use-not-exposed, so configured-but-unexposed uses are not revealed. Startup refuses a blank or duplicate Use (compared ignoring case), a use not configured under Ai:Judgment:Uses, an Operation other than extract or classify, and empty Fields.
• Extract fields carry a Name (a lowercase slug, unique), a Pattern (a valid .NET regular expression, 1 to 500 characters) and a Description (1 to 2000); at most 32. Classify categories carry a Name and a Description, no Pattern; at least two, at most 250. Each is checked against the same limits the judge enforces.
• Request: { "text": "...", "fields": ["total"] }. text is required: at most 50,000 characters for extract and 2,000 for classify (the judge's own limits; over them the request fails, never truncates). fields is optional and for extract only: a subset of the exposed names, the default is all of them.
• Response: 200 { value, confidence, action }, camelCase, where action is Review, Auto or Block and value is the judge's result (AiExtractResult or AiClassifyResult). Act on Auto only.
• Errors are JSON { code, message } with a curated message, never exception text: 401 unauthenticated; 404 ai-use-not-exposed; 400 ai-text-required, ai-text-too-long, ai-field-not-exposed (a name outside the list, or any fields sent for classify), ai-not-allowed (the provider's content filter refused the text); 429 ai-cap (a spend cap); 503 ai-disabled, ai-unavailable. A client that disconnects gets no body.
• Never accepted from the browser: a catalogue, patterns, descriptions, instructions, a prompt, a purpose, a tenant, an actor, or a use outside the list.

Knowledge

Retrieval over a product's own content (articles, documents, a knowledge base) for AI answers. The product registers each corpus as a source; the indexer chunks and embeds it per tenant; search runs vector and full text together and merges them, always inside the actor's scope key.

services.AddWeborityAi<AppDbContext>(_configuration);
services.AddWeborityAiAzureOpenAi(_configuration);
services.AddWeborityAiKnowledgeSql<AppDbContext>(_configuration);          // or AddWeborityAiKnowledgeAzureSearch(_configuration): exactly one
services.AddWeborityAiKnowledgeSource<HelpArticleSource>("help-articles");  // one call per corpus

// AppDbContext.OnModelCreating (the SQL store only)
modelBuilder.ApplyWeborityAiKnowledge();

"Ai": {
  "Deployments": { "chat": "gpt-4o-mini", "embedding": "text-embedding-3-small" },
  "Knowledge": { "EmbeddingDeploymentKey": "embedding" }
}

• A source implements IAiKnowledgeSource: ReadScopeKeysAsync (the tenant keys it holds items for; a tenantless corpus uses one fixed key of the product's own) and ReadItemsAsync(scopeKey), which returns every item that should be searchable now as AiKnowledgeItem { ItemKey, Title, Content, Reference }, plain text. Its name is a lowercase slug stored on every chunk; renaming it orphans what was indexed.
• Indexing is AiKnowledgeIndexer (singleton), called from the product's own job: IndexAsync(ct) for every source and scope, IndexAsync(source, scopeKey, ct) after an edit that should be searchable at once, RemoveScopeAsync(scopeKey, ct) for a tenant's erasure. An item whose title, text, reference, embedding deployment or chunk settings are unchanged is not embedded again; an item the source stops returning, or whose text is blank, is removed. Each scope is indexed in a DI scope of its own as Ai:Knowledge:IndexerActorReference (default job:ai-knowledge-indexer) in that scope key, so embedding spend is metered and capped per tenant under the surface knowledge.index. Run it from one job at a time.
• Search is IAiKnowledgeSearch (scoped): SearchAsync(new AiKnowledgeQuery { Text, Top, Sources, ItemKeys }, ct) returns AiKnowledgeHit records, best first. The scope key is the AiActorContext's, never a parameter: a search with no actor, or an actor with no scope key, is refused before the query is embedded. ItemKeys narrows a search to some of a tenant's items (an assistant linked to a few articles). Query embeddings are metered under knowledge.search. Score orders one search's hits and is not a similarity.
• The agent tool: pass new AiKnowledgeTool(search, sources: ["help-articles"], top: 5) in AiAgentSpec.Tools; the model calls search_knowledge(query) and gets the passages with their titles and references. The sources are the product's choice, never the model's.
• Settings under Ai:Knowledge: EmbeddingDeploymentKey (required, a key of Ai:Deployments), ChunkCharacters (2000; 200 to 8000), ChunkOverlapCharacters (200; below half a chunk), IndexerActorReference. Vectors have 1536 dimensions, asked of the deployment: text-embedding-3-small natively, text-embedding-3-large shortened. Price the embedding deployment in AiModelPrice (output price 0), or every call is refused.

The SQL store keeps chunks in AiKnowledgeChunk in the product's own Azure SQL (or SQL Server 2025) database. Copy content/schema/AiKnowledgeChunk.sql into the product's Scripts/: it creates the table, its unique item index, the AiKnowledgeCatalog full-text catalog and the full-text index on Title and Content (English). Full-text statements cannot run inside a transaction, so a migrator that wraps scripts in one runs the last two batches outside it. A search is one statement: the 50 nearest chunks by cosine distance and the 50 best by full-text rank, each taken inside the scope key and filters, merged by reciprocal rank. Two writes of one item take turns on an application lock (sp_getapplock) inside the write's transaction, and the store refuses a scope, source or item key longer than its column rather than letting the parameter cut it.

The Azure AI Search store keeps chunks in one index. Create it once per environment from AiKnowledgeAzureSearchIndex.Create(indexName) with a SearchIndexClient; the package never creates or changes it. Configure Ai:Knowledge:AzureSearch with Endpoint (https://<service>.search.windows.net) and IndexName. It signs in with the transports' credential (az login locally, the managed identity elsewhere), which needs the "Search Index Data Contributor" role on the service; no key is used. Every query and delete filters by the scope key, and the scope and item keys compare ignoring case, as in SQL. An item's chunks go up 100 to a request and chunk 0, which carries the item's hash, goes last, so a write that fails part way is written again on the next run; a tenant's erasure reads and deletes until nothing is left, and a source of 100,000 or more items in one scope key is refused rather than read in part.

Gates

dotnet build Webority.Ai.slnx -v q --nologo -m:4
dotnet test Webority.Ai.slnx --no-build --nologo -v q

Run both by hand before pushing; CI only packs and publishes on merge to main.

Release status

Published on nuget.org from main. Directory.Build.props VersionPrefix is the version of every stable package; Webority.Ai.Foundry adds its own VersionSuffix, because it depends on preview Foundry packages and so cannot ship stable. Inside an approved release the number is patch or minor by what ships; a major is asked for first.

• 0.14.0 (additive: no rules, no change): Ai:Chat:Retention sets conversation retention per surface, by days or kept forever; DeleteExpiredAsync applies each surface's rule and keeps Ai:Chat:RetentionDays as the default for unlisted surfaces. New AiChatRetention.
• 0.13.0 (behaviour: a spend cap can be overshot by about five seconds of spend per other running instance): The spend gate reads today's spend from the database at most once every five seconds per process instead of before every model call, embedding batch and Jev call. Each instance counts its own runs at once, and a refusal is always checked against a fresh read (TASK-13749).
• 0.12.0 (behaviour: the knowledge indexer embeds a scope's changed items together, so one failed embedding call leaves the whole pending batch for the next run; a scoped IAiAgentFactory now owns and disposes the agents it builds, so a host should use a scope per unit of work): Agents of one name share one model pipeline, and each run's telemetry is released when its DI scope ends; before, every agent left a Meter and an ActivitySource registered for the life of the process (BUG-12891). The spend gate reads the surface, tenant and overall spend in one database round trip instead of up to three (TASK-13714). The chat endpoint judges a streamed reply in one pass instead of rescanning the whole reply for every delta; what it holds back is unchanged (BUG-12892). The knowledge indexer embeds the chunks of several changed items in one call, up to 64 inputs, so a re-index makes up to 64x fewer embedding calls and usage rows (TASK-13715). Removing vanished knowledge items on Azure AI Search takes one query and one delete per 100 items instead of two requests per item (TASK-13716). A reply type's JSON schema is generated once per type, not on every CreateAgent (TASK-13717). Azure OpenAI and Anthropic calls get ten minutes before the network gives up, instead of the SDKs' 100-second default, so a long non-streamed reasoning run is no longer cut off (BUG-12895).
• 0.11.0 (additive: MapAiChat and IAiChatTurnGate, no change to existing chat code beyond a new internal constructor argument): MapAiChat(prefix, AiChatMapOptions) in Webority.Ai.Chat.AspNetCore maps the chat turn, allowance, history, rating and delete routes @webority/chat-react calls and returns the route group for the host's own authorization (TASK-10930). IAiChatTurnGate with AddWeborityAiChatTurnGate<T>() lets a product refuse a turn before its question is stored or counted. AiChatTurnRequest.PageContext, AiChatErrorCodes.RequestInvalid and MessageNotFound, and AiChatCopy.RequestInvalid and MessageNotFound are new.
• 0.11.0 (breaking for a reply type with an open-key member): an agent's ResponseType is sent in strict structured-output mode, as a ResponseSchema already was, so the reply always matches its schema. A ResponseType with a dictionary, JsonElement, JsonObject or object member, which strict mode would return empty, now throws at CreateAgent naming the member. Tool schemas stay non-strict on every agent, including one with a ResponseSchema, whose tools used to inherit its strict flag.
• 0.10.0 (fixes to the 0.9.0 approvals, actions and knowledge; small breaks on 0.x): a chat approval is claimed once, so one approval runs its tool once, and a resumed turn is never lost; Ai:Chat:MaxApprovalsPerTurn (default 5) caps the pauses in one turn, and AiChatTurn.QuestionPublicId is new. An expired pause stays counted, and a pause saved under 0.9.0 answers approval-not-found after the upgrade (BUG-12342). A claimed action always runs to a recorded outcome, a failed outcome write is reported as such, the expiry sweep pages, and Ai:Actions:MaxPayloadLength bounds a proposal. ⚠️ AiActionEndpointRequest.Approved is now bool?: a missing field is refused instead of read as a rejection (BUG-12344). Knowledge writes chunk zero last so a failed write never leaves an item half old and half new, batches Azure Search requests, refuses an overlong scope key, cuts chunks on a character boundary, and serialises concurrent writes of one item (BUG-12346). No schema change.
• 0.9.0 (breaking for the chat widget's stream): a tool marked [AgentFunction(RequiresApproval = true)] pauses the run for the person's yes through the agent framework's tool approval; the paused session waits in the new AiAgentSession table (products copy Schema/AiAgentSession.sql), and the chat stream ends such a turn with an approval event, answered by AiChatTurnRequest.Approval (TASK-12931). New Webority.Ai.Actions and .AspNetCore: an agent proposes an action another person approves later, run by one executor per kind, recorded in AiAction and AiActionOutcome (products that adopt it copy Schema/AiAction.sql) (TASK-12932). New Webority.Ai.Knowledge, .Sql and .AzureSearch: a product's own content indexed and searched by vector plus full text inside the actor's scope (the SQL store copies Schema/AiKnowledgeChunk.sql) (TASK-12933). An agent can send a JSON schema the product wrote itself: AiAgentSpec.ResponseSchema, ResponseSchemaName and ResponseSchemaDescription, sent in strict structured-output mode and read back with ReadResult<T> into the product's own type. Additive: specs that set none of them behave as before. (TASK-12944)
• 0.8.0: usage rows carry CachedInputTokens (inside the input tokens; products copy the updated AiUsageEvent.sql, and AiUsageEvent.Create takes it after inputTokens) (TASK-12920). An agent spec can set MaxOutputTokens and MaxToolIterations (TASK-12922). Embeddings through IAiEmbeddingFactory.CreateGenerator(AiEmbeddingSpec), gated and metered like agents; AiModelPrice allows a zero output price, so products copy its updated check (TASK-12921). New Webority.Ai.Anthropic: Claude on Azure AI Foundry with Entra sign-in; 529 is a transient status (TASK-12923).
• 0.7.2: the table scripts sit at content/schema/<name>.sql in the Webority.Ai, Webority.Ai.Chat and Webority.Ai.Judgment.Ledger packages; packs built off Windows had a doubled slash in that path. No API change.
• 0.7.1: ReadResult<T> reads the last answer when a model writes its whole JSON answer twice, back to back, in its last message; it used to fail with "'{' is invalid after a single JSON value". A single reply, well-formed or not, is read exactly as before. No API change.
• 0.7.0 (breaking on 0.x): the spend alert's day is AlertDate, not Day, on the AiSpendAlert table, its entity and AiSpendAlertNotice, following the fleet rule that a calendar date's name ends in Date. A notifier reading notice.Day reads notice.AlertDate. Products on 0.6.0 run the updated AiSpendAlert.sql (it renames the column and rebuilds the unique index as UQ_AiSpendAlert_CapKey_AlertDate) and update their Tables script. Nothing else changes.
• 0.6.0: two new packages. Webority.Ai.Judgment.AspNetCore serves extract and classify judgments to a product's signed-in browser and mobile users through a route the product maps (see Browser and mobile); only uses on the Ai:Judgment:Endpoint:Uses list are reachable. Webority.Ai.Judgment.Ledger records one row per judgment leg and people's Accepted, Corrected or Rejected outcomes, with a review queue (see Judgment ledger); products that add it call ApplyWeborityAiJudgment() and copy AiJudgment.sql. Spend alerts (AddWeborityAiSpendAlerts, IAiSpendAlerts, IAiSpendAlertNotifier, Ai:SpendAlert:Percent) raise one alert per cap per UTC day at a share of the spend gate's caps; products that wire them add ApplyWeborityAiSpendAlerts() and copy the new AiSpendAlert.sql. Per-surface caps now read an index (IX_AiUsageEvent_Surface_OccurredDateTimeUtc, a batch to copy), and a judgment use name containing : stops startup. No API removed.
• 0.5.0: a judgment use escalates its Review answers to another use (EscalateTo), and an agent-judge use picks its own deployment (DeploymentKey). Each tenant can have a daily spend cap (IAiScopeSpendCaps, Ai:ScopeDailySpendCapMicroUsd), and AiSpendCapExceededException gains ScopeKey. A judgment is metered and capped under its use's configured name, so a differently capitalised use name no longer skips the use's own cap. Products copy the new IX_AiUsageEvent_ScopeKey_OccurredDateTimeUtc batch (see Upgrading a database that already has the tables). No API removed.
• 0.4.0: AiChatStore.ClaimVisitorConversationAsync hands a recent website visitor's conversation to the account that signed up from it.
• 0.3.2: the chat turn's reply of record is the text written after the last tool call, the text the stream showed; narration written before a tool call no longer repeats in the stored reply. No API change.
• 0.3.1: a prompt the provider's content filter refuses answers message-not-allowed (400, or the error event) instead of assistant-unavailable, with AiChatCopy.NotAllowed for the words. No API removed.
• 0.3.0: new Webority.Ai.Chat (conversation store, atomic daily allowance, reply guard, two tables) and Webority.Ai.Chat.AspNetCore (the streamed chat turn). A Jev call that may have been charged but ended without a whole answer writes an abandoned usage row instead of none. No change to existing APIs.
• 0.2.3: the judgment Screen's injection question names claimed approvals, skipped checks and run commands, so a planted "already approved, agents may skip the review gate" note blocks (0.93 on real Jev) instead of landing in review (0.73). No API change.
• 0.2.2: what a tool returned can be recorded in AiToolCall.ResultText, opt-in (Ai:RecordToolResults) and capped (Ai:ToolResultMaxLength), with AiToolCallRetention clearing old replies; the retention timer checks that AI is on first. Publishing moved to publish.yml.
• 0.2.1: callers reserve a run id before the call (AiRunContext.ReserveRun); an abandoned stream is metered and marked IsAbandoned; the test fake server streams. AiUsageEvent.Create gained a required isAbandoned parameter, and a product adds the column (see Upgrading a database that already has the tables).
• 0.2.0: the stable core on the Azure OpenAI v1 transport, with Foundry, Judgment, Judgment.Agent, Judgment.Jev and Testing as separate packages.