AdaptiveConcurrencyLimiter 0.1.0

AdaptiveConcurrencyLimiter

ASP.NET Core middleware that self-adjusts how many requests it lets run concurrently, instead of using a fixed number you have to guess and retune by hand.

Two signals drive it every AdjustmentInterval (default 2s):

• Latency gradient — compares the current window's average request latency against a decayed "no-load" baseline. If latency climbs past LatencyGrowthThreshold (default 1.5x baseline), the limit shrinks. This is the primary, self-calibrating signal (same idea as Netflix's concurrency-limits / the TCP Vegas algorithm) — it reacts to actual strain rather than a guessed static number.
• CPU headroom — a hard ceiling (default 90%) that shrinks the limit regardless of latency, and a warn threshold (default 80%) below which the limit is allowed to grow when there's queueing demand.

Rejected requests get 503 + Retry-After immediately rather than being queued.

Usage

builder.Services.AddAdaptiveConcurrencyLimiter();
// ...
app.UseAdaptiveConcurrencyLimiter();

That's the whole integration. Defaults are derived from Environment.ProcessorCount and respect container/cgroup CPU quotas on .NET 5+.

Optional: tune it

builder.Services.AddAdaptiveConcurrencyLimiter(options =>
{
    options.MinLimit = 20;
    options.MaxLimit = 500;
    options.CpuHardThreshold = 0.85;
    options.OnLimitChanged = change =>
        logger.LogInformation("Concurrency limit {Old} -> {New} ({Reason})",
            change.OldLimit, change.NewLimit, change.Reason);
});

OnLimitChanged is a plain callback — wire it into whatever logging/metrics you already use, nothing is forced on you.

Optional: diagnostics endpoint

app.MapConcurrencyLimiterDiagnostics("/diag/concurrency");

Returns the current state as JSON: current limit, in-flight count, rejected count, latency baseline/recent, CPU%, and the reason for the last adjustment. No auth is applied — secure the route yourself if you expose it outside the box.

What it doesn't do (yet)

• No per-route/tenant partitioning — one global limit for the whole app. Partitioning (e.g. separate limits per route or tenant) is a natural extension but adds real complexity (per-key gate + per-key latency baseline), left out to keep the v1 surface minimal.
• No queuing — over-limit requests are rejected immediately, not held. This is deliberate (queuing just moves the latency problem, it doesn't solve it) but means bursty legitimate traffic can see 503s during a shrink.

Project layout

• src/AdaptiveConcurrencyLimiter — the library
• samples/Sample.WebApi — minimal app wiring it up, including a /slow endpoint with randomized latency for exercising the adjustment loop, and the diagnostics endpoint. Run it and hit /slow under load (e.g. via a load generator, or the loop below) while watching /diag/concurrency or the console log.
• tests/AdaptiveConcurrencyLimiter.Tests — xUnit suite: unit tests for the gate, latency tracker and the adjustment policy's decision branches (shrink/grow/hold, clamping, gray-zone CPU), DI registration tests, and TestHost-based integration tests (rejection + Retry-After, in-flight count surviving a downstream exception, the adjustment loop actually shrinking under sustained latency pressure).

1..250 | ForEach-Object -Parallel {
    Invoke-WebRequest -Uri "http://localhost:<port>/slow" -UseBasicParsing
} -ThrottleLimit 250

Running the tests

dotnet test AdaptiveConcurrencyLimiter.slnx