PollyOpenAI 1.0.0

PollyOpenAI

Polly v8 resilience for OpenAI and Azure OpenAI API calls.
Automatic retry with exponential back-off, Retry-After header support, circuit breaker, and per-request timeout — wired up in one line via IHttpClientBuilder.

Why PollyOpenAI?

OpenAI and Azure OpenAI APIs regularly return transient errors in production:

Installation

dotnet add package PollyOpenAI

Quick Start

// Program.cs
builder.Services.AddHttpClient("openai", client =>
    {
        client.BaseAddress = new Uri("https://api.openai.com/");
        client.DefaultRequestHeaders.Add("Authorization", $"Bearer {apiKey}");
    })
    .AddPollyOpenAIResilience();

// Inject IHttpClientFactory and pass to the OpenAI SDK
var httpClient = httpClientFactory.CreateClient("openai");
var openAIClient = new OpenAIClient(new ApiKeyCredential(apiKey),
    new OpenAIClientOptions { Transport = new HttpClientTransport(httpClient) });

Azure OpenAI

builder.Services.AddHttpClient("azure-openai", client =>
    {
        client.BaseAddress = new Uri("https://my-resource.openai.azure.com/");
    })
    .AddPollyOpenAIResilience(options =>
    {
        options.MaxRetries = 5;
        options.BaseDelay = TimeSpan.FromSeconds(2);
        options.MaxDelay = TimeSpan.FromSeconds(60);
    });

Named client shortcut

// Registers IHttpClientFactory with a client named "PollyOpenAI"
builder.Services.AddPollyOpenAIHttpClient(
    baseAddress: new Uri("https://api.openai.com/"),
    configure: options => options.MaxRetries = 4);

// Resolve
var client = httpClientFactory.CreateClient("PollyOpenAI");

Configuration

.AddPollyOpenAIResilience(options =>
{
    // Retry
    options.MaxRetries = 3;                          // default: 3
    options.BaseDelay = TimeSpan.FromSeconds(1);     // default: 1 s
    options.MaxDelay = TimeSpan.FromSeconds(30);     // default: 30 s
    options.RespectRetryAfterHeader = true;          // default: true

    // Circuit breaker
    options.CircuitBreakerMinimumThroughput = 5;     // default: 5
    options.CircuitBreakerFailureRatio = 0.5;        // default: 0.5 (50%)
    options.CircuitBreakerSamplingDuration = TimeSpan.FromSeconds(30);
    options.CircuitBreakerBreakDuration = TimeSpan.FromSeconds(30);

    // Timeout (per request attempt)
    options.RequestTimeout = TimeSpan.FromSeconds(100);  // default: 100 s

    // Customise which status codes trigger retry
    options.TransientStatusCodes = new HashSet<HttpStatusCode>
    {
        HttpStatusCode.TooManyRequests,     // 429
        HttpStatusCode.InternalServerError, // 500
        HttpStatusCode.BadGateway,          // 502
        HttpStatusCode.ServiceUnavailable,  // 503
        HttpStatusCode.GatewayTimeout,      // 504
    };
})

How it works

The pipeline order (outer → inner):

Request
  └─► Retry (exponential back-off, honours Retry-After)
        └─► Circuit Breaker (opens on sustained failures)
              └─► Timeout (per-attempt)
                    └─► HttpClient → OpenAI API

1. Retry — on OpenAITransientException (thrown for any status code in TransientStatusCodes), waits for the Retry-After delta if present, otherwise uses exponential back-off with jitter. The request body is buffered before the first send so it can be re-transmitted on every attempt.

2. Circuit Breaker — after enough failures within the sampling window the circuit opens, all subsequent calls fail-fast with BrokenCircuitException until the break duration elapses. Prevents hammering a degraded API.

3. Timeout — enforces a per-attempt deadline; throws TimeoutRejectedException if the OpenAI API doesn't respond in time.

Exception types

Comparison

Related Packages

Package	Downloads	Description
PollyChaos		Chaos engineering / fault injection for Polly v8
PollyMediatR		Polly v8 pipelines for MediatR request handlers
PollyEFCore		Polly v8 resilience for EF Core queries and SaveChanges
PollyHealthChecks		ASP.NET Core health checks for Polly v8 circuit breakers
PollyBackoff		Custom back-off strategies for Polly v8

License

MIT