LlmCostGuard 0.1.0

LLM Cost Guard for .NET

Real-time cost tracking, budget enforcement, and usage analytics for LLM applications in .NET.

Features

• Real-Time Cost Tracking: Track costs per request with automatic pricing lookup
• Budget Enforcement: Set daily, weekly, or monthly limits with warning thresholds
• Multi-Provider Support: OpenAI, Anthropic, AWS Bedrock, Google Vertex AI
• Flexible Storage: In-memory (default), SQLite, or custom backends
• Cost Attribution: Tag requests by team, feature, or user for cost allocation
• Caching Support: Track cache hits and calculate savings
• Rate Limiting: Sliding window rate limiters per model, provider, or global
• Hierarchical Tracking: Group related LLM calls with spans
• Audit Logging: Compliance-ready audit trails for budget and config changes
• Metrics Export: Prometheus-compatible metrics exporter

Installation

dotnet add package LlmCostGuard

Quick Start

using LlmCostGuard;
using LlmCostGuard.Models;

// Create a cost tracker
using var tracker = new CostTracker();

// Add a daily budget with warning at 80%
tracker.AddBudget("daily", limit: 100.00m, period: BudgetPeriod.Day);

// Track an API call
var record = await tracker.TrackAsync(
    provider: "openai",
    model: "gpt-4o",
    inputTokens: 1000,
    outputTokens: 500,
    tags: new Dictionary<string, string>
    {
        ["team"] = "platform",
        ["feature"] = "chat"
    }
);

Console.WriteLine($"Cost: ${record.TotalCost:F4}");

Budget Enforcement

using var tracker = new CostTracker();

// Warn when approaching limit
tracker.AddBudget(new Budget
{
    Name = "team-ml",
    Limit = 500.00m,
    Period = BudgetPeriod.Month,
    Action = BudgetAction.Warn,
    WarningThreshold = 0.8m,
    Tags = new Dictionary<string, string> { ["team"] = "ml" }
});

// Block requests when exceeded
tracker.AddBudget(new Budget
{
    Name = "per-request",
    Limit = 1.00m,
    Period = BudgetPeriod.Request,
    Action = BudgetAction.Block
});

// Subscribe to budget events
tracker.BudgetWarning += (sender, e) =>
{
    Console.WriteLine($"Warning: {e.Status.Budget.Name} at {e.Status.UtilizationPercent:F1}%");
};

tracker.BudgetExceeded += (sender, e) =>
{
    Console.WriteLine($"Exceeded: {e.Status.Budget.Name}");
};

Cost Attribution

// Track costs by team and feature
await tracker.TrackAsync(
    provider: "anthropic",
    model: "claude-3-5-sonnet",
    inputTokens: 2000,
    outputTokens: 1000,
    tags: new Dictionary<string, string>
    {
        ["team"] = "platform",
        ["feature"] = "summarization",
        ["user_id"] = "user-123"
    }
);

// Get costs grouped by team
var report = await tracker.GetReportAsync(
    startDate: DateTime.UtcNow.Date,
    groupBy: new[] { "team" }
);

foreach (var group in report.Groups!)
{
    Console.WriteLine($"{group.GroupBy["team"]}: ${group.Cost:F2}");
}

Pre-Request Budget Check

// Check if a request would exceed budget before making the API call
var estimatedCost = 0.05m;
var (allowed, blockingBudget) = await tracker.CheckBudgetAsync(
    estimatedCost,
    tags: new Dictionary<string, string> { ["team"] = "ml" }
);

if (!allowed)
{
    Console.WriteLine($"Request blocked: {blockingBudget!.Budget.Name} would be exceeded");
    // Handle blocked request (queue, fallback model, etc.)
}

Reports

// Get daily report
var report = await tracker.GetReportAsync(
    startDate: DateTime.UtcNow.Date,
    endDate: DateTime.UtcNow.Date.AddDays(1)
);

Console.WriteLine($"Total Cost: ${report.TotalCost:F2}");
Console.WriteLine($"Total Calls: {report.TotalCalls}");
Console.WriteLine($"Success Rate: {(double)report.SuccessfulCalls / report.TotalCalls * 100:F1}%");

// Get costs by provider and model
var detailedReport = await tracker.GetReportAsync(
    startDate: DateTime.UtcNow.AddDays(-7),
    groupBy: new[] { "provider", "model" }
);

Supported Models

OpenAI

• gpt-4o, gpt-4o-mini
• gpt-4-turbo, gpt-4, gpt-3.5-turbo
• o1, o1-mini, o1-preview
• text-embedding-3-large, text-embedding-3-small

Anthropic

• claude-3-5-sonnet, claude-3-5-haiku
• claude-3-opus, claude-3-sonnet, claude-3-haiku

AWS Bedrock

• Claude models via Bedrock
• Amazon Titan models
• Meta Llama models

Google Vertex AI

• gemini-1.5-pro, gemini-1.5-flash
• gemini-1.0-pro

Rate Limiting

using LlmCostGuard.RateLimiting;

using var limiter = new RateLimiter();

// Global rate limit: 100 requests per minute
limiter.AddLimit("global", 100, RateLimitPeriod.Minute, RateLimitScope.Global);

// Per-model rate limit: 10 requests per second
limiter.AddLimit("per-model", 10, RateLimitPeriod.Second, RateLimitScope.Model);

// Check and record a request
var result = limiter.Record(provider: "openai", model: "gpt-4o");

if (!result.Allowed)
{
    Console.WriteLine($"Rate limited by {result.ExceededLimitName}");
    Console.WriteLine($"Retry after {result.RetryAfterSeconds:F1} seconds");
}

Hierarchical Tracking (Spans)

using LlmCostGuard.Tracking;

// Create a span to group related LLM calls
using var span = new Span("document-processing");
span.Start();

// Track multiple calls within the span
span.RecordCall(0.05m, 1000, 500, "gpt-4o");
span.RecordCall(0.03m, 800, 400, "gpt-4o-mini");

// Access aggregated metrics
Console.WriteLine($"Total cost: ${span.TotalCost:F4}");
Console.WriteLine($"Total calls: {span.CallCount}");
Console.WriteLine($"Models used: {string.Join(", ", span.ModelsUsed)}");

// Nested spans
using var childSpan = new Span("summarization");
childSpan.Start();
childSpan.RecordCall(0.02m, 500, 200, "gpt-4o-mini");
// Child costs automatically propagate to parent when span ends

Audit Logging

using LlmCostGuard.Audit;

// Create an audit backend
var auditBackend = new MemoryAuditBackend();
// Or: var auditBackend = new FileAuditBackend("audit.log");
// Or: var auditBackend = new ConsoleAuditBackend();

// Log audit events
auditBackend.Log(new AuditEvent
{
    EventType = AuditEventType.BudgetCreated,
    Actor = "admin",
    Resource = "daily-budget",
    Details = new Dictionary<string, object> { ["limit"] = 100.0 }
});

// Query audit events
var events = auditBackend.Query(
    eventType: AuditEventType.BudgetExceeded,
    startDate: DateTime.UtcNow.AddDays(-7),
    limit: 50
);

Metrics Export

using LlmCostGuard.Metrics;

// Create a Prometheus exporter
var exporter = new PrometheusExporter();

// Record metrics
exporter.Counter(MetricNames.RequestCount, 1, new Dictionary<string, string>
{
    ["provider"] = "openai",
    ["model"] = "gpt-4o"
});
exporter.Histogram(MetricNames.Latency, 150.0);
exporter.Gauge(MetricNames.BudgetUtilization, 75.5);

// Export in Prometheus format (for scraping endpoint)
string prometheusOutput = exporter.Export();

// Or use the logging exporter for debugging
var loggingExporter = new LoggingExporter();
loggingExporter.Counter("requests", 1);

SQLite Backend

using LlmCostGuard.Backends;

// Use SQLite for persistent storage
var backend = new SqliteBackend("costs.db");
// Or: var backend = new SqliteBackend("sqlite:///costs.db");

using var tracker = new CostTracker(backend: backend);

// All cost records are now persisted to SQLite
await tracker.TrackAsync("openai", "gpt-4o", 1000, 500);

Custom Pricing

var customPricing = new Dictionary<string, ModelPricing>
{
    ["custom/my-model"] = new ModelPricing
    {
        InputCostPer1K = 0.001m,
        OutputCostPer1K = 0.002m
    }
};

var pricingLoader = new PricingLoader(customPricing);
using var tracker = new CostTracker(pricing: pricingLoader);

License

MIT License - see LICENSE for details.

Related Projects

• llm-cost-guard (Python) - Python version with the same API design