Samhammer.ThreadPoolStarvationLogger 10.0.1

Samhammer.ThreadPoolStarvationLogger

Detects .NET ThreadPool starvation in a running application and logs the details needed to actually diagnose it, instead of just a vague symptom.

Why

.NET's ThreadPool starts with a minimum thread count derived from Environment.ProcessorCount — which, in a container with a CPU limit (e.g. Kubernetes resources.limits.cpu: "1"), can be as low as 1. When demand briefly exceeds that, the pool grows slowly by design, and any code waiting for a free thread (including framework internals like ASP.NET Core's health check middleware) can stall for seconds. This typically shows up as sporadic, hard-to-reproduce timeouts or TaskCanceledExceptions with no obvious cause.

This library queues a trivial no-op work item to the ThreadPool on a fixed interval and measures how long it actually took to get dispatched. When that delay exceeds a configurable threshold, it logs a detailed snapshot to help you tell starvation apart from other causes:

• ThreadCount before/after — did the pool grow during the stall, or was it already sitting flat?
• PendingWorkItems before/after — was there a backlog, or did the queue start empty and pile up?
• AvailableWorker / AvailableIocp vs. their configured min/max — how much headroom the pool thinks it has.
• CpuThrottledMs — on cgroup v2 hosts (e.g. Kubernetes on Linux), the CPU time lost to CFS quota throttling during that exact delay window, read directly from /sys/fs/cgroup/cpu.stat. This lets you tell ThreadPool starvation apart from the container simply not getting CPU time at all — two problems that look identical from the outside but need different fixes.

Install

dotnet add package Samhammer.ThreadPoolStarvationLogger

Usage

using Samhammer.ThreadPoolStarvationLogger;

builder.Services.AddThreadPoolStarvationLogger();

With custom options:

builder.Services.AddThreadPoolStarvationLogger(options =>
{
    options.PollInterval = TimeSpan.FromMilliseconds(200);
    options.WarnThreshold = TimeSpan.FromSeconds(1);
});

Or via configuration binding, like any other options class:

builder.Services.Configure<ThreadPoolStarvationLoggerOptions>(builder.Configuration.GetSection("ThreadPoolStarvationLogger"));
builder.Services.AddThreadPoolStarvationLogger();

Options

Example log output

ThreadPool dispatch delay of 4039ms detected. CpuThrottledMs=0.4 ThreadCount 7->7 PendingWorkItems 0->23 AvailableWorker=32762/32767 (min 1) AvailableIocp=1000/1000 (min 1)

Reading it: the queue was empty right before the delay (PendingWorkItems 0->23), the thread count never grew (ThreadCount 7->7), and almost none of the 4 seconds was CPU throttling (CpuThrottledMs=0.4) — pointing at ThreadPool scheduling itself, not the container's CPU allocation, as the cause.

What this does not fix

This is a diagnostic tool, not a mitigation. If it confirms starvation, the standard fix is raising the ThreadPool's minimum thread count above the CPU-limit-derived default, e.g.:

ThreadPool.SetMinThreads(20, 20);

set as early as possible during application startup. See the .NET docs on ThreadPool.SetMinThreads for the tradeoffs involved.

License

MIT

Contribute

How to publish package

• create git tag
• The nuget package will be published automatically by a github action