Resilience Patterns

Polly-based retry, circuit breaker, and timeout policies for HTTP clients and external services.

SCRIPE uses Polly resilience policies for all external HTTP calls and service integrations. The three core policies retry with exponential backoff, circuit breaker, and timeout protect against transient failures and cascading outages.

Resilience Architecture

Resilience Policy Pipeline

HTTP Request

Timeout Policy

Retry Policy

Circuit Breaker

External Service

Retry Policy

Retry Policy ConfigurationC#

/// <summary>
/// Configures retry with exponential or linear backoff + jitter.
/// Only retries on transient HTTP errors (5xx, 408, network errors).
/// </summary>
services.AddResilientHttpClient("InventoryService", configuration);

// Under the hood:
builder.AddRetry(new HttpRetryStrategyOptions
{
    MaxRetryAttempts = options.Retry.MaxRetryAttempts,        // Default: 3
    BackoffType = options.Retry.BackoffType == "Exponential"
        ? DelayBackoffType.Exponential
        : DelayBackoffType.Linear,
    Delay = TimeSpan.FromMilliseconds(options.Retry.BaseDelayMs), // Default: 500ms
    MaxDelay = TimeSpan.FromMilliseconds(options.Retry.MaxDelayMs), // Default: 30s
    UseJitter = options.Retry.UseJitter,                      // Default: true
    ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
        .HandleResult(r => (int)r.StatusCode >= 500)
        .HandleResult(r => r.StatusCode == HttpStatusCode.RequestTimeout)
        .Handle<HttpRequestException>(),
});

Attempt	Exponential Delay	Linear Delay	With Jitter
1st retry	500ms	500ms	500ms ± 250ms
2nd retry	1,000ms	1,000ms	1,000ms ± 500ms
3rd retry	2,000ms	1,500ms	2,000ms ± 1,000ms
4th retry	4,000ms	2,000ms	4,000ms ± 2,000ms

Circuit Breaker

The circuit breaker prevents repeated calls to a failing service. After 5 consecutive failures, it opens and rejects requests for 30 seconds, then transitions to half-open to test if the service has recovered.

Circuit Breaker ConfigurationC#

builder.AddCircuitBreaker(new CircuitBreakerStrategyOptions<HttpResponseMessage>
{
    FailureRatio = options.CircuitBreaker.FailureRatio,      // Default: 0.5 (50%)
    MinimumThroughput = options.CircuitBreaker.MinThroughput, // Default: 10
    SamplingDuration = TimeSpan.FromSeconds(
        options.CircuitBreaker.SamplingDurationSeconds),       // Default: 30s
    BreakDuration = TimeSpan.FromSeconds(
        options.CircuitBreaker.BreakDurationSeconds),          // Default: 30s
    ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
        .HandleResult(r => (int)r.StatusCode >= 500)
        .Handle<HttpRequestException>(),
});

State	Behavior	Transitions To
Closed (Normal)	Requests pass through normally	Open (when failure ratio > threshold)
Open (Tripped)	Requests fail-fast immediately (no call)	Half-Open (after break duration)
Half-Open (Testing)	One test request allowed through	Closed (on success) / Open (on failure)

Timeout Policy

Timeout PolicyC#

builder.AddTimeout(TimeSpan.FromSeconds(
    options.Timeout.TimeoutSeconds       // Default: 30
));

// When timeout triggers:
// - CancellationToken is cancelled
// - TimeoutRejectedException is thrown
// - Retry policy may retry the request

Configuration

appsettings.json — Resilience ConfigurationJSON

{
  "Resilience": {
    "Retry": {
      "MaxRetryAttempts": 3,
      "BackoffType": "Exponential",
      "BaseDelayMs": 500,
      "MaxDelayMs": 30000,
      "UseJitter": true
    },
    "CircuitBreaker": {
      "FailureRatio": 0.5,
      "MinThroughput": 10,
      "SamplingDurationSeconds": 30,
      "BreakDurationSeconds": 30
    },
    "Timeout": {
      "TimeoutSeconds": 30
    }
  }
}

PreviousBackground Jobs NextGateway & Deployment