CostGuard 0.1.0

CostGuard

Cost tracking, budgets, and automatic model fallback for Microsoft.Extensions.AI.

Microsoft.Extensions.AI tells you how many tokens a request used. CostGuard turns that into money — and lets you act on it:

• 💵 Per-request cost in USD — computed from a built-in (and fully overridable) pricing catalog for Anthropic, OpenAI, and Google models, attached to every ChatResponse.
• 🚧 Budgets — daily / monthly / all-time limits per tenant, per user, or globally. When a limit is hit: throw, log, or automatically degrade to a cheaper model.
• 🏷️ Scopes — attribute spend to tenants/users/features via an ambient CostScope, a ChatOptions key, or a custom selector.
• 📈 OpenTelemetry-friendly metrics — cost and token counters on the CostGuard meter.
• 🔌 Plain middleware — one .UseCostGuard() call in your existing ChatClientBuilder pipeline. Works with any IChatClient provider (OpenAI, Anthropic, Azure, Ollama, …), streaming included.

Install

dotnet add package CostGuard

Quickstart

using CostGuard;
using Microsoft.Extensions.AI;

IChatClient client = new ChatClientBuilder(innerProviderClient)
    .UseCostGuard(o =>
    {
        // Each scope (tenant/user) gets $10/day; past that, requests run on a cheaper model.
        o.Budgets.Add(new Budget
        {
            Scope = CostScopes.PerScope,
            Period = BudgetPeriod.Daily,
            LimitUsd = 10m,
            Action = BudgetAction.DegradeModel,
            FallbackModelId = "claude-haiku-4-5",
        });

        // Hard stop at $500/month across the whole app.
        o.Budgets.Add(new Budget
        {
            Scope = CostScopes.Total,
            Period = BudgetPeriod.Monthly,
            LimitUsd = 500m,
            Action = BudgetAction.Throw,
        });
    })
    .Build();

// Attribute spend to a tenant for everything inside the scope:
using (CostScope.Begin($"tenant:{tenantId}"))
{
    ChatResponse response = await client.GetResponseAsync(messages);

    ChatCost? cost = response.GetCostGuardCost();
    Console.WriteLine($"This request cost ${cost?.TotalUsd:0.####}");
}

No budgets configured? CostGuard still meters every request — useful as a pure cost observability layer.

How budgets work

Budgets are checked before each request against spend already recorded in the store, so a scope can overshoot its limit by at most one request. Windows are fixed UTC calendar periods (Daily, Monthly, AllTime).

Scopes

Three ways to say who is spending, in priority order:

// 1. A custom selector (e.g. read your own request context):
o.ScopeSelector = options => myRequestContext.TenantId;

// 2. Per-request, on ChatOptions:
var options = new ChatOptions
{
    AdditionalProperties = new() { [CostGuardKeys.Scope] = "user:42" },
};

// 3. Ambient, flows across awaits (great for ASP.NET Core middleware):
using (CostScope.Begin("tenant:acme")) { ... }

Unscoped requests land in CostScopes.Default ("global").

Pricing catalog

PricingCatalog.CreateDefault() ships best-effort list prices (snapshot: July 2026, USD per 1M tokens) for current Anthropic (Claude), OpenAI (GPT-5.x), and Google (Gemini) models, including discounted cache-read rates where known. Model ids resolve by exact match first, then longest prefix — so claude-sonnet-5-20260115 finds the claude-sonnet-5 entry.

Provider prices change. For anything billing-critical, verify against your invoice and override:

o.Pricing.Add("my-fine-tune", inputPerMTokUsd: 4m, outputPerMTokUsd: 20m);

// Or load/override from a JSON feed you control:
o.Pricing.AddFromJson("""
{
  "claude-sonnet-5": { "inputPerMTok": 2.0, "outputPerMTok": 10.0, "cachedInputPerMTok": 0.2 },
  "anthropic.claude-opus-5": { "inputPerMTok": 5.0, "outputPerMTok": 25.0 }
}
""");

Unknown models are logged once and skipped by default (UnknownModelBehavior.Warn); set Throw to fail fast or Ignore to silence.

Reporting

CostGuardOptions? costGuard = client.GetCostGuardOptions();
decimal monthly = await costGuard!.SpendStore.GetCurrentSpendAsync("tenant:acme", BudgetPeriod.Monthly);
decimal total   = await costGuard.SpendStore.GetCurrentSpendAsync(CostScopes.Total, BudgetPeriod.AllTime);

The default InMemorySpendStore is per-process and resets on restart. For durable, multi-instance enforcement, implement the two-method ISpendStore interface over your database or Redis and assign it to o.SpendStore.

Metrics

Register the meter with OpenTelemetry:

services.AddOpenTelemetry().WithMetrics(m => m.AddMeter(CostGuardDiagnostics.MeterName));

Notes & limitations

• Costs are estimates computed from reported token usage and configured prices — reconcile against provider invoices for accounting purposes.
• Cached-input discounts treat cached tokens as a subset of input tokens (OpenAI-style reporting); detection reads common provider keys from UsageDetails.AdditionalCounts.
• Budget enforcement is pre-flight: parallel requests in the same scope can each pass the check before either records spend.
• Streaming responses are metered when the provider reports usage in the stream (most do, on the final update).

License

MIT