Middleware Pipeline

11 middleware components in execution order from exception handling through tenant context to field projection.

SCRIPE's HTTP request pipeline consists of 11 middleware components executed in a specific order. Each middleware has a single responsibility and can short-circuit the pipeline on failure. Understanding the order is critical for debugging and extending the system.

Pipeline Overview

Requests flow through the middleware pipeline from top to bottom. Each middleware can process the request, modify it, or short-circuit by returning a response directly. The order matters tenant context must be established before any tenant-scoped operation.

Full HTTP Request Middleware Execution Pipeline
PHASE 1: Request Bootstrapping & Diagnostics
25. Request Logging & Audit — Serilog audit trail creation
1. Forwarded Headers (Real IP)
PHASE 5: Endpoint Mapping & Execution
2. Global Exception Middleware — Sanitizes stack traces
26. Health Checks (/health)
3. HSTS & Security Headers — HSTS in Prod, X-Frame-Options
27. ETag Middleware — Calculates ETag hashes
4. Correlation ID Middleware — Assigns X-Correlation-Id
28. API Controllers / SignalR / YARP
5. Observability Middleware — OpenTelemetry enrichment
6. HTTPS Redirection
7. Response Compression
8. Request Localization — Culture selection
PHASE 2: Routing, Gateway & Access Policies
9. Static Files (/api/files)
10. CORS Middleware — Origin checking
11. Cloudflare Origin Verification — Fail-closed check
12. Background Jobs Dashboard
13. Rate Limiter — Tiered rate limiting
14. Response Caching & Cache Headers
PHASE 3: Authentication & Anti-CSRF
15. Cookie Auth (Cookie-to-Bearer)
16. Authentication (JWT) — Validates access token
17. CSRF Protection — Double-submit validation
18. Input Sanitization — Strips HTML tags
PHASE 4: Authorization, Replay & Tenant Isolation
19. Authorization (RBAC) — Permission checks
20. Must Change Password Gate — Blocks expired credentials
21. Replay Protection — Timestamp ±5m & Nonce unique
22. Tenant Context Validation — Validates tenant context
23. TenantContextMiddleware — IDataScopeService scoping
24. Field Projection — Filters restricted fields

1. Global Exception Handler

Catches all unhandled exceptions and returns structured JSON error responses. In development, includes stack traces. In production, returns generic error messages to prevent information leakage.

GlobalExceptionMiddleware.csC#
/// <summary>
/// Top-level exception handler. Catches ALL unhandled exceptions
/// and returns a structured error response instead of exposing stack traces.
/// Must be FIRST in the pipeline to catch errors from all middleware.
/// </summary>
public class GlobalExceptionMiddleware
{
    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (ValidationException ex)
        {
            context.Response.StatusCode = 400;
            await context.Response.WriteAsJsonAsync(new
            {
                type = "ValidationError",
                errors = ex.Errors.Select(e => new { e.PropertyName, e.ErrorMessage })
            });
        }
        catch (UnauthorizedAccessException)
        {
            context.Response.StatusCode = 401;
            await context.Response.WriteAsJsonAsync(new
            {
                type = "Unauthorized",
                message = "Authentication required"
            });
        }
        catch (ForbiddenAccessException ex)
        {
            context.Response.StatusCode = 403;
            await context.Response.WriteAsJsonAsync(new { type = "Forbidden", ex.Message });
        }
        catch (NotFoundException ex)
        {
            context.Response.StatusCode = 404;
            await context.Response.WriteAsJsonAsync(new { type = "NotFound", ex.Message });
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Unhandled exception for {Path}", context.Request.Path);
            context.Response.StatusCode = 500;
            await context.Response.WriteAsJsonAsync(new
            {
                type = "InternalError",
                message = _env.IsDevelopment() ? ex.Message : "An error occurred",
                traceId = Activity.Current?.Id ?? context.TraceIdentifier
            });
        }
    }
}

2. Correlation ID

Generates or reads an X-Correlation-ID header for distributed tracing. The same ID is attached to all log entries, audit records, and downstream API calls for the request.

CorrelationIdMiddleware.csC#
/// <summary>
/// Assigns a unique correlation ID to every request for distributed tracing.
/// If the client sends X-Correlation-Id, it is reused; otherwise generated.
/// The correlation ID is:
///   1. Added to the response header
///   2. Stored in HttpContext.Items
///   3. Added to the logging scope (appears in all log entries)
/// </summary>
public class CorrelationIdMiddleware
{
    private const string HeaderName = "X-Correlation-Id";

    public async Task InvokeAsync(HttpContext context)
    {
        var correlationId = context.Request.Headers[HeaderName].FirstOrDefault()
            ?? Guid.NewGuid().ToString();

        context.Items["CorrelationId"] = correlationId;
        context.Response.Headers[HeaderName] = correlationId;

        using (_logger.BeginScope(new Dictionary<string, object>
        {
            ["CorrelationId"] = correlationId
        }))
        {
            await _next(context);
        }
    }
}

3. Request Logging

Logs request metadata (method, path, status, duration) with Serilog structured logging. Sensitive paths (login, password) have their body masked to prevent credential leakage in logs.

RequestLoggingMiddleware.csC#
/// <summary>
/// Logs every HTTP request with structured data:
/// Method, Path, StatusCode, Duration, UserId, TenantId, IP, UserAgent.
/// Also creates audit log entries for state-changing requests.
/// </summary>
public class RequestLoggingMiddleware
{
    public async Task InvokeAsync(HttpContext context, IAuditService auditService)
    {
        var stopwatch = Stopwatch.StartNew();
        var originalBody = context.Response.Body;

        try
        {
            await _next(context);
        }
        finally
        {
            stopwatch.Stop();
            var statusCode = context.Response.StatusCode;
            var method = context.Request.Method;
            var path = context.Request.Path;
            var userId = context.User.FindFirst("sub")?.Value;
            var tenantId = context.User.FindFirst("tenant_id")?.Value;

            _logger.LogInformation(
                "{Method} {Path} → {StatusCode} ({Duration}ms) | User:{UserId} Tenant:{TenantId} IP:{IP}",
                method, path, statusCode, stopwatch.ElapsedMilliseconds,
                userId, tenantId, context.Connection.RemoteIpAddress);

            // Create audit entry for non-GET requests
            if (method != "GET" && userId != null)
            {
                await auditService.LogAsync(new AuditEntry
                {
                    Action = $"{method} {path}",
                    UserId = userId,
                    TenantId = tenantId,
                    StatusCode = statusCode,
                    Duration = stopwatch.ElapsedMilliseconds,
                    IpAddress = context.Connection.RemoteIpAddress?.ToString(),
                    UserAgent = context.Request.Headers.UserAgent.ToString()
                });
            }
        }
    }
}

Reads the access_token cookie and injects it as a Bearer token in the Authorization header. This allows the frontend to use HttpOnly cookies while maintaining JWT-based authentication.

CookieAuthMiddleware.csC#
/// <summary>
/// Fallback authentication middleware that reads JWT from HttpOnly cookie
/// when the Authorization header is missing.
/// This supports browser-based flows where cookies are more secure than
/// storing tokens in localStorage.
/// </summary>
public class CookieAuthMiddleware
{
    private const string CookieName = "scripe-auth";

    public async Task InvokeAsync(HttpContext context)
    {
        // Only activate if no Authorization header present
        if (!context.Request.Headers.ContainsKey("Authorization"))
        {
            var token = context.Request.Cookies[CookieName];
            if (!string.IsNullOrEmpty(token))
            {
                // Set Authorization header from cookie
                context.Request.Headers.Authorization = $"Bearer {token}";
            }
        }
        await _next(context);
    }
}

5. Tenant Context

Extracts the tenant_id claim from the JWT token and sets the current tenant in IDataScopeService. All subsequent database queries are automatically scoped to this tenant via EF Core global query filters.

7. Field Projection

Removes restricted fields from JSON responses based on the current user's role permissions. Uses the restrictedFields configuration to filter out sensitive properties before the response is sent to the client.

FieldProjectionMiddleware.csC#
/// <summary>
/// Filters response JSON to only include fields the user's role permits.
/// Uses RestrictedFields from role configuration.
/// Runs AFTER the controller has produced the response.
/// </summary>
public class FieldProjectionMiddleware
{
    public async Task InvokeAsync(HttpContext context)
    {
        // Capture the response body
        var originalBodyStream = context.Response.Body;
        using var memoryStream = new MemoryStream();
        context.Response.Body = memoryStream;

        await _next(context);

        // Read the response
        memoryStream.Seek(0, SeekOrigin.Begin);
        var responseBody = await new StreamReader(memoryStream).ReadToEndAsync();

        // Apply field filtering if restricted fields are set
        if (context.Items.TryGetValue("RestrictedFields", out var fields)
            && fields is HashSet<string> restrictedFields)
        {
            responseBody = FilterJsonFields(responseBody, restrictedFields);
        }

        // Write filtered response
        var bytes = Encoding.UTF8.GetBytes(responseBody);
        await originalBodyStream.WriteAsync(bytes);
    }
}

6. Cache Headers

Sets appropriate Cache-Control headers based on response type. API responses use no-cache, no-store. Static files use max-age with ETag validation. Uploaded files use tenant-specific cache keys.

CacheHeadersMiddleware.csC#
/// <summary>
/// Implements conditional caching with ETag support:
/// 1. Computes ETag (MD5 hash of response body)
/// 2. Handles If-None-Match → returns 304 Not Modified
/// 3. Sets Cache-Control headers based on endpoint configuration
/// </summary>
public class CacheHeadersMiddleware
{
    public async Task InvokeAsync(HttpContext context)
    {
        await _next(context);

        if (context.Request.Method == "GET" &&
            context.Response.StatusCode == 200)
        {
            // Compute ETag from response body
            var etag = ComputeETag(context.Response);
            context.Response.Headers.ETag = etag;

            // Check If-None-Match from client
            var clientETag = context.Request.Headers.IfNoneMatch.FirstOrDefault();
            if (clientETag == etag)
            {
                context.Response.StatusCode = 304; // Not Modified
                context.Response.ContentLength = 0;
                return;
            }
        }
    }
}

Observability Middleware

Collects request metrics (duration, status codes, error rates) and exposes them via a /metrics endpoint for Prometheus scraping. Includes distributed tracing with OpenTelemetry integration.

ObservabilityMiddleware.csC#
/// <summary>
/// Enriches distributed traces and metrics for monitoring:
/// 1. Adds custom tags to Activity (OpenTelemetry span)
/// 2. Records request duration histogram
/// 3. Increments request counter by status code
/// </summary>
public class ObservabilityMiddleware
{
    private static readonly Histogram<double> RequestDuration =
        Meters.Default.CreateHistogram<double>("http.request.duration", "ms");

    private static readonly Counter<long> RequestCount =
        Meters.Default.CreateCounter<long>("http.request.count");

    public async Task InvokeAsync(HttpContext context)
    {
        var stopwatch = Stopwatch.StartNew();
        var activity = Activity.Current;

        activity?.SetTag("tenant.id", context.User.FindFirst("tenant_id")?.Value);
        activity?.SetTag("user.id", context.User.FindFirst("sub")?.Value);

        try
        {
            await _next(context);
        }
        finally
        {
            stopwatch.Stop();
            var tags = new TagList
            {
                { "http.method", context.Request.Method },
                { "http.status_code", context.Response.StatusCode },
                { "http.route", context.GetEndpoint()?.DisplayName }
            };

            RequestDuration.Record(stopwatch.Elapsed.TotalMilliseconds, tags);
            RequestCount.Add(1, tags);
        }
    }
}

Middleware Registration Order

The middleware registration order in Program.cs determines execution order. Changing the order can break functionality for example, registering TenantContext before Authentication would fail because the JWT hasn't been validated yet.

Program.cs — Middleware Pipeline RegistrationC#
var app = builder.Build();

// ─── Middleware Pipeline (ORDER MATTERS!) ───────────────
app.UseForwardedHeaders();                           // 1. Resolve client IP
app.UseMiddleware<GlobalExceptionMiddleware>();      // 2. Catch all errors
if (!app.Environment.IsDevelopment())
    app.UseHsts();                                   // 3. Enable HSTS
app.UseSecurityHeaders();                            // 4. Set secure response headers
app.UseCorrelationId();                              // 5. Trace logging correlation
app.UseObservabilityMiddleware();                    // 6. Trace enrichment
if (!app.Environment.IsDevelopment())
    app.UseHttpsRedirection();                       // 7. Enforce SSL
app.UseResponseCompression();                        // 8. Brotli/Gzip compression
app.UseRequestLocalization();                        // 9. Multi-lingual context
app.UseStaticFiles();                                // 10. Serve static files
app.UseCorsConfiguration();                          // 11. CORS policies
app.UseCloudflareOriginVerification();               // 12. Cloudflare gate
app.UseBackgroundJobsDashboard();                    // 13. Hangfire dashboard
app.UseRateLimiting();                               // 14. Throttling limiters
app.UseResponseCaching();                            // 15. Response caching
app.UseMiddleware<CacheHeadersMiddleware>();          // 16. Cache headers
app.UseCookieAuth();                                 // 17. Cookie auth fallback
app.UseAuthentication();                             // 18. Authenticate JWT token
app.UseCsrfProtection();                             // 19. CSRF double-submit
app.UseInputSanitization();                          // 20. Input sanitization
app.UseAuthorization();                              // 21. Check RBAC permissions
app.UseMiddleware<MustChangePasswordMiddleware>();   // 22. MCP flag check
app.UseReplayProtection();                           // 23. Timestamp & Nonce check
app.UseTenantContextValidation();                    // 24. Switch-tenant checks
app.UseMiddleware<FieldProjectionMiddleware>();       // 25. Field level security
app.UseRequestLogging();                             // 26. Log HTTP audits (after auth)

app.MapHealthChecks("/health");                      // 27. Health endpoint
app.UseMiddleware<ETagMiddleware>();                 // 28. Client ETag validation
app.MapControllers();                                // 29. Route to controllers
app.Run();

Middleware Summary

#MiddlewarePurpose & ResponsibilityShort-Circuits?Scope & Conditions
1Forwarded HeadersResolves client IP/Host headers behind reverse proxiesNoAll incoming requests
2GlobalExceptionMiddlewareCatches all unhandled exceptions and returns structured errorsOn exceptionAll requests (top handler)
3HSTSEnforces Strict Transport Security header in productionNoProduction requests only
4Security HeadersInjects anti-clickjacking, nosniff, and CSP headersNoAll responses
5Correlation IDGenerates or extracts X-Correlation-Id for distributed tracingNoAll requests
6Observability MiddlewareEnriches trace spans with tenant and user ID metadataNoAll requests
7HTTPS RedirectionRedirects unencrypted HTTP requests to secure HTTPSYes (Redirects 301)Non-development requests
8Response CompressionCompresses HTTP payload using Brotli/GzipNoResponses exceeding threshold
9Request LocalizationSets request culture context based on Accept-LanguageNoAll requests
10Static FilesServes secure files from FileHost folder under /api/filesYes (200 / 304)Requests to RequestPath
11CORSValidates cross-origin requests against whitelisted originsYes (400 / Preflight 204)Cross-origin requests
12Cloudflare Origin VerificationBlocks non-health requests that bypass Cloudflare proxiesYes (403 Forbidden)Non-health requests in Prod
13Background Jobs DashboardProvides secure access dashboard for Hangfire or QuartzYes (UI pages / 401)/hangfire or dashboard routes
14Rate LimitingEnforces request throttling based on IP/User policiesYes (429 Too Many Requests)All endpoints (configured)
15Response CachingServes cached responses directly when availableYes (Cache hit 200)Configured GET requests
16CacheHeadersMiddlewareInjects cache-control headers and computes ETag hashesYes (304 Not Modified)All GET requests
17CookieAuthMiddlewareReads JWT from httpOnly cookie as Authorization header fallbackNoAuthorization header missing
18AuthenticationValidates JWT signature, validity, and extracts identity claimsYes (401 Unauthorized on invalid)All requests
19CSRF ProtectionValidates double-submit cookie signature for state mutationYes (403 CSRF Mismatch)POST/PUT/DELETE non-Bearer
20Input SanitizationStrips dangerous HTML/script tags from incoming JSON modelsNoMutating requests (JSON bodies)
21AuthorizationValidates user permissions and roles against endpoint targetsYes (403 Forbidden)Endpoints with auth attributes
22MustChangePasswordMiddlewareBlocks resource access when password needs rotationYes (403 Force Change)JWT contains mcp=true claim
23Replay ProtectionValidates uniqueness of request UUID nonces within skew windowYes (409 Conflict / 400)Mutating authenticated requests
24Tenant Context ValidationEnsures request tenant matches user bounds or drill-downsYes (403 Forbidden)Tenant-scoped requests
25TenantContextMiddlewareBinds the active tenant context to IDataScopeServiceNoTenant-scoped requests
26FieldProjectionMiddlewareRemoves unauthorized restricted fields from JSON responsesNoGET requests with active policies
27Request LoggingLogs request auditing data including user and tenant IDsNoAll requests (post-auth context)
28Health ChecksExposes system health, database, and readiness probesYes (200 / 503)/health and /health/ready
29ETag MiddlewareValidates If-None-Match header against response MD5 hashYes (304 Not Modified)GET requests