PollyOpenAI 1.0.3

PollyOpenAI

Polly v8 resilience for OpenAI and Azure OpenAI API calls.
Automatic retry with exponential back-off, Retry-After header support, circuit breaker, and per-request timeout — wired up in one line via IHttpClientBuilder.

Why PollyOpenAI?

OpenAI and Azure OpenAI APIs regularly return transient errors in production:

Installation

dotnet add package PollyOpenAI

Quick Start

// Program.cs
builder.Services.AddHttpClient("openai", client =>
    {
        client.BaseAddress = new Uri("https://api.openai.com/");
        client.DefaultRequestHeaders.Add("Authorization", $"Bearer {apiKey}");
    })
    .AddPollyOpenAIResilience();

// Inject IHttpClientFactory and pass to the OpenAI SDK
var httpClient = httpClientFactory.CreateClient("openai");
var openAIClient = new OpenAIClient(new ApiKeyCredential(apiKey),
    new OpenAIClientOptions { Transport = new HttpClientTransport(httpClient) });

Azure OpenAI

builder.Services.AddHttpClient("azure-openai", client =>
    {
        client.BaseAddress = new Uri("https://my-resource.openai.azure.com/");
    })
    .AddPollyOpenAIResilience(options =>
    {
        options.MaxRetries = 5;
        options.BaseDelay = TimeSpan.FromSeconds(2);
        options.MaxDelay = TimeSpan.FromSeconds(60);
    });

Named client shortcut

// Registers IHttpClientFactory with a client named "PollyOpenAI"
builder.Services.AddPollyOpenAIHttpClient(
    baseAddress: new Uri("https://api.openai.com/"),
    configure: options => options.MaxRetries = 4);

// Resolve
var client = httpClientFactory.CreateClient("PollyOpenAI");

Configuration

.AddPollyOpenAIResilience(options =>
{
    // Retry
    options.MaxRetries = 3;                          // default: 3
    options.BaseDelay = TimeSpan.FromSeconds(1);     // default: 1 s
    options.MaxDelay = TimeSpan.FromSeconds(30);     // default: 30 s
    options.RespectRetryAfterHeader = true;          // default: true

    // Circuit breaker
    options.CircuitBreakerMinimumThroughput = 5;     // default: 5
    options.CircuitBreakerFailureRatio = 0.5;        // default: 0.5 (50%)
    options.CircuitBreakerSamplingDuration = TimeSpan.FromSeconds(30);
    options.CircuitBreakerBreakDuration = TimeSpan.FromSeconds(30);

    // Timeout (per request attempt)
    options.RequestTimeout = TimeSpan.FromSeconds(100);  // default: 100 s

    // Customise which status codes trigger retry
    options.TransientStatusCodes = new HashSet<HttpStatusCode>
    {
        HttpStatusCode.TooManyRequests,     // 429
        HttpStatusCode.InternalServerError, // 500
        HttpStatusCode.BadGateway,          // 502
        HttpStatusCode.ServiceUnavailable,  // 503
        HttpStatusCode.GatewayTimeout,      // 504
    };
})

How it works

The pipeline order (outer → inner):

Request
  └─► Retry (exponential back-off, honours Retry-After)
        └─► Circuit Breaker (opens on sustained failures)
              └─► Timeout (per-attempt)
                    └─► HttpClient → OpenAI API

1. Retry — on OpenAITransientException (thrown for any status code in TransientStatusCodes), waits for the Retry-After delta if present, otherwise uses exponential back-off with jitter. The request body is buffered before the first send so it can be re-transmitted on every attempt.

2. Circuit Breaker — after enough failures within the sampling window the circuit opens, all subsequent calls fail-fast with BrokenCircuitException until the break duration elapses. Prevents hammering a degraded API.

3. Timeout — enforces a per-attempt deadline; throws TimeoutRejectedException if the OpenAI API doesn't respond in time.

Exception types

Comparison

Related Packages

Package	Downloads	Description
PollyHealthChecks		ASP.NET Core health checks for Polly v8 circuit breakers — expose circuit-breaker state (Closed, HalfOpen, Open, Isolated) as /health endpoint responses
PollyBackoff		Backoff delay strategies for Polly v8 resilience pipelines
PollyGrpc		Polly v8 resilience interceptor for gRPC
PollyEFCore		Polly v8 resilience pipelines for Entity Framework Core — wrap every EF Core query and SaveChanges with retry, timeout and circuit-breaker via a single AddPollyResilience() call
PollyRabbitMQ		Polly v8 resilience for RabbitMQ.Client v7+ — retry, circuit-breaker, and timeout for IChannel operations, with built-in RabbitMqTransientErrors predicate covering AlreadyClosedException, BrokerUnreachableException, OperationInterruptedException, and ConnectFailureException
PollyMailKit		Polly v8 resilience pipelines for MailKit — retry, timeout, and circuit-breaker for SmtpClient.SendAsync and any MailKit SMTP operation
PollyMassTransit		Polly v8 resilience pipelines for MassTransit — retry, timeout, and circuit-breaker for IBus.Publish and ISendEndpointProvider.Send
PollyAzureEventHub		Polly v8 resilience pipelines for Azure Event Hubs — retry, timeout, and circuit-breaker for EventHubProducerClient and EventHubConsumerClient
PollySignalR		Polly v8 reconnect policy for SignalR
PollyElasticsearch		Polly v8 resilience pipelines for Elastic.Clients.Elasticsearch 8+ — retry, timeout, and circuit-breaker for any Elasticsearch operation, plus a built-in ElasticTransientErrors predicate covering rate limiting (429), service unavailability (503), gateway timeouts (504), and connection failures
PollyHangfire		Polly v8 resilience pipelines for Hangfire — retry, timeout, and circuit-breaker for IBackgroundJobClient.Enqueue and Schedule
PollySendGrid		Polly v8 resilience pipelines for SendGrid — retry, timeout, and circuit-breaker for ISendGridClient.SendEmailAsync
PollyMediatR		Polly v8 resilience pipelines for MediatR — add retry, timeout, circuit-breaker, rate-limiting, hedging, and chaos engineering to any MediatR request handler with a single line of DI registration
PollyAzureKeyVault		Polly v8 resilience pipelines for Azure Key Vault — retry, timeout, and circuit-breaker for SecretClient, KeyClient, and CertificateClient
PollyAzureQueueStorage		Polly v8 resilience pipelines for Azure Queue Storage — retry, timeout, and circuit-breaker for Azure.Storage.Queues QueueClient
PollyRedis		Polly v8 resilience for StackExchange.Redis
PollyAzureServiceBus		Polly v8 resilience for Azure Service Bus — retry, circuit breaker, and timeout for sending and receiving messages
PollyKafka		Polly v8 resilience for Confluent.Kafka — retry, circuit breaker, and timeout for producers and consumers
PollyAzureTableStorage		Polly v8 resilience pipelines for Azure Table Storage — retry, timeout, and circuit-breaker for Azure.Data.Tables TableClient
PollyChaos		Chaos engineering and fault-injection resilience strategies for Polly v8 pipelines

💼 Need .NET consulting?

The author of this package is available for consulting on Polly v8 resilience, Azure cloud architecture, and clean .NET design.

→ solidqualitysolutions.com · LinkedIn

License

MIT