AiTokenTracker 0.4.0

AiTokenTracker .NET SDK

AiTokenTracker captures outbound LLM traffic and sends it to your ingest API for centralized token/cost parsing.

This SDK supports three integration paths so teams can adopt with minimal refactoring:

1. HttpClient Handler Integration (preferred)
   • Use .AddAiTokenTracker() on IHttpClientFactory clients.
   • Captures full request/response envelopes.
2. SDK Wrapper Integration
   • Use IAiTokenTrackerSdkClient when provider SDK calls bypass your HttpClient pipeline.
   • Captures full request/response payloads from DTOs.
3. Diagnostics Fallback
   • Captures metadata-only for LLM calls not covered by #1 or #2.
   • Useful for visibility and gap detection.

Installation

dotnet add package AiTokenTracker --version <latest>

Quick Start

Register once in DI:

using AiTokenTracker;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddAiTokenTracker(options =>
{
    options.AuthToken = "your-subscription-or-api-key";
    options.EnableDiagnosticsFallback = true;
});

Then for preferred full-fidelity capture on HTTP clients you own:

builder.Services
    .AddHttpClient("OpenAiHttp")
    .AddAiTokenTracker();

Integration Modes

1) HttpClient Handler Integration (Preferred)

Best when your outbound LLM calls go through IHttpClientFactory.

builder.Services.AddHttpClient("LlmClient")
    .AddAiTokenTracker();

Any classified LLM request sent by this client is captured with full body.

Scoped Custom Filters For Automatic Capture

You can attach custom filters to automatic handler/diagnostics capture by opening an interception scope in the current async flow:

var scope = AiTokenTrackerInterceptionScope.Create(new Dictionary<string, string>
{
    ["Workflow"] = "AutoPublish",
    ["JobId"] = "job-42"
});

scope.Open();
try
{
    using HttpClient llmClient = httpClientFactory.CreateClient("LlmClient");
    using HttpResponseMessage response = await llmClient.PostAsync("/v1/responses", content, cancellationToken);
}
finally
{
    scope.Close();
}

You can also use using:

using var scope = AiTokenTrackerInterceptionScope.Create(new Dictionary<string, string>
{
    ["Workflow"] = "AutoPublish"
});
scope.Open();
await llmClient.PostAsync("/v1/responses", content, cancellationToken);

Use one scope per logical request/workflow and always close in finally to avoid leaking context.

2) SDK Wrapper Integration (For SDKs that bypass HttpClientFactory)

If a provider SDK hides transport internals, wrap calls with IAiTokenTrackerSdkClient. providerHint is optional and should be passed when known. method is optional and defaults to "POST" when omitted.

public sealed class TrackedOpenAiClient
{
    private readonly OpenAIClient _inner;
    private readonly IAiTokenTrackerSdkClient _tracker;

    public TrackedOpenAiClient(OpenAIClient inner, IAiTokenTrackerSdkClient tracker)
    {
        _inner = inner;
        _tracker = tracker;
    }

    public async Task<Response> CreateResponseAsync(string model, string userPrompt, CancellationToken cancellationToken)
    {
        var responseClient = _inner.GetOpenAIResponseClient(model);

        var requestPayload = new
        {
            userInputText = userPrompt
        };

        IAiTokenTrackerCallScope scope = _tracker.BeginLlmCall(
            request: requestPayload,
            providerHint: "openai");

        try
        {
            var response = await responseClient.CreateResponseAsync(userPrompt, cancellationToken: cancellationToken);
            _ = scope.CompleteAsync(response.Value, 200, cancellationToken: CancellationToken.None);
            return response.Value;
        }
        catch (Exception ex)
        {
            _ = scope.FailAsync(ex, cancellationToken: CancellationToken.None);
            throw;
        }
    }
}

This pattern guarantees capture even when provider SDKs do not run through your tracked HttpClient handlers.

3) Diagnostics Fallback

Enabled by default:

options.EnableDiagnosticsFallback = true;

The fallback listens to System.Net.Http diagnostics and records likely LLM calls not captured by the first two integrations.

Important:

• Fallback is generally metadata-only (method/url/status/headers).
• It is intended for visibility and missed-call detection, not primary full-fidelity ingest.

Services Registered

AddAiTokenTracker(...) registers:

• AiTokenTrackerDelegatingHandler
• IAiTokenTrackerIngestionClient
• IAiTokenTrackerSdkClient
• ILlmRequestClassifier default implementation (DefaultLlmRequestClassifier)
• Diagnostics fallback hosted service
• Named ingest client: AiTokenTracker.IngestionClient

Configuration

AiTokenTrackerOptions:

• AuthToken (string, required)
• EnableDiagnosticsFallback (bool, default: true)

Custom Provider Classification

The SDK includes DefaultLlmRequestClassifier heuristics (OpenAI/Anthropic/Gemini/Unify/common paths).

To customize, implement ILlmRequestClassifier:

public interface ILlmRequestClassifier
{
    bool TryClassify(HttpRequestMessage request, out string? providerHint);
}

Register your classifier in DI. It will run alongside defaults.

Choosing a Strategy

Use this guideline:

• If call path uses HttpClientFactory: use Handler Integration.
• If call path uses provider SDK abstraction: use Wrapper Integration.
• Keep Diagnostics Fallback on to identify uncovered traffic.

Recommended rollout for large codebases:

1. Enable SDK + diagnostics fallback globally.
2. Add .AddAiTokenTracker() to known LLM HttpClient registrations.
3. Add wrapper integration only at central SDK service boundaries.
4. Use diagnostics events to locate remaining uncovered flows.

Troubleshooting

No ingest events received

Check:

1. AuthToken is configured and non-empty.
2. Request is classified as LLM traffic.
3. If using provider SDKs, confirm wrapper integration is applied where the call is made.

Calls are visible but token fields are zero

That usually means payload shape was not full-fidelity or not parseable by server analyzers. Ensure handler/wrapper path is providing request/response bodies.

Duplicate events

If both handler and wrapper are applied to the exact same call path, you can ingest duplicates. Use one primary integration path per call flow.

Notes

• The SDK intentionally sends envelopes asynchronously to avoid blocking LLM call latency.