ChannelExtensions.Durability 2026.6.22.1

ChannelExtensions.Durability

Durable variants of System.Threading.Channels.Channel<T> that survive process restarts and back-pressure without dropping or reordering items.

Each durable channel is a drop-in Channel<T>: you write through Writer and read through Reader exactly as with an in-memory channel. The durability is transparent to producers and consumers.

Available channels

More durable channels (e.g. other backing stores) can be added alongside FileBackedChannel. Each lives under ChannelExtensions.Durability with its own Create… factory and options type.

FileBackedChannel

An in-memory bounded channel that spills to disk when it fills, then drains the disk backlog back into the channel once the consumer catches up.

How it works

• Writes normally go straight to an in-memory bounded channel (direct mode).
• When the channel fills, the channel switches to spill mode: every write is appended to disk instead. This preserves order — new items can never jump ahead of items already queued on disk.
• A background writer batches spilled items into newline-delimited JSON (.ndjson) blocks. A block is committed when it reaches MaxBlockSize items or CommitInterval elapses, whichever comes first.
• A background reader replays committed blocks oldest-first back into the channel, waiting for space so the consumer is never overwhelmed.
• Once the entire disk backlog is drained, the channel reverts to direct mode.

Guarantees

• No drops — items that don't fit in memory are persisted, not discarded.
• Ordering — preserved across the spill boundary and across restarts.
• Crash recovery — on startup, half-written blocks left by a crash are finalized (the truncated trailing record is dropped) and replayed. If a backlog exists at startup, the channel begins in spill mode so new writes queue behind it.
• Idempotent replay — each block tracks how many records have been handed to the consumer via a checkpoint written before each emit, so a crash mid-replay does not re-deliver an already-emitted record (at-most-once at the boundary; no duplicates).
• Durable writes — blocks and checkpoints are written with FileOptions.WriteThrough, bypassing the OS write cache.

Usage

using System.Threading.Channels;
using ChannelExtensions.Durability;
using ChannelExtensions.Durability.FileBackedChannel;

// Create the channel. The path is created (and verified read/write/delete-able)
// in the constructor; an unusable path throws.
Channel<MyEvent> channel = Channel.CreateFileBackedChannel<MyEvent>(
    new FileBackedChannelOptions(capacity: 10_000, path: @"C:\data\my-channel"));

// Producer — identical to any Channel<T>.
await channel.Writer.WriteAsync(new MyEvent(...));

// Consumer — identical to any Channel<T>.
await foreach (var item in channel.Reader.ReadAllAsync())
{
    Handle(item);
}

Dependency injection

The factory runs eagerly, so register it with a factory delegate when you want the configured ILogger (or other services) injected:

builder.Services.AddSingleton<Channel<MyEvent>>(sp =>
    Channel.CreateFileBackedChannel<MyEvent>(
        new FileBackedChannelOptions(10_000, @"C:\data\my-channel")
        {
            Logger = sp.GetRequiredService<ILoggerFactory>().CreateLogger("FileBackedChannel"),
        }));

Note: a durable channel is not an IHostedService. Its background drain loops start in the constructor. Logging is configured purely by the Logger option — when omitted it defaults to a no-op logger.

Options

FileBackedChannelOptions extends ChannelOptions.

On-disk layout

All files live under Path:

Logging

All log events carry an EventId (see FileBackedChannelEventIds): spill started/completed, backlog recovered, block commit/replay/recovery failures, quarantine/delete failures, and drain-loop faults. Failures are logged and handled rather than thrown out of the background loops.

Disposal

Disposing the channel cancels the background loops and gives in-flight commits a bounded chance to finish. Items still buffered in memory (not yet spilled) are lost on a hard process kill, which is the window the spill + recovery machinery exists to minimize.

Project layout

Running the tests

dotnet test ChannelExtensions.Durability.Tests