RequestBatcher 0.0.2

RequestBatcher

RequestBatcher lets application code submit one request and await one Task, while an application handler receives multiple queued requests as an IReadOnlyList<TRequest>.

Batching does not require callers to assemble a collection. Separate callers can each submit one TRequest concurrently. RequestBatcher coalesces requests already queued for the same partition into handler batches of up to BatchSize. ProcessAsync(IEnumerable<TRequest>) is an additional submission option, not a prerequisite for batching.

Full documentation | 简体中文

When to Use It

Use RequestBatcher for independent database, cache, or downstream operations that benefit from a batch API. It also fits short traffic bursts that need bounded internal queue capacity and downstream concurrency, related requests that benefit from partition-local order, and work where caller cancellation should remove only requests that have not yet been dispatched. Dispatched work is allowed to finish independently of that caller.

For database updates, if partial success within one handler invocation would leave inconsistent state, the handler should execute that invocation in one transaction. RequestBatcher propagates the handler outcome but cannot roll back writes that have already been committed.

When Not to Use It

Do not use RequestBatcher when work must survive process failure, remain inside the caller's transaction, return a direct TResult, wait for a minimum batch size, or rely on automatic retries or exactly-once effects. It is also not suitable when an in-flight downstream operation must stop as soon as its individual caller disconnects, times out, or cancels. One handler batch can contain requests from several callers, so caller cancellation tokens are not forwarded to the handler and cannot cancel the shared handler call.

Install

dotnet add package RequestBatcher

Usage

Define a request and a handler for one batch:

public sealed record OrderWriteRequest(long OrderId, decimal Amount);

public interface IOrderStore
{
    Task WriteBatchAsync(
        IReadOnlyList<OrderWriteRequest> requests,
        CancellationToken cancellationToken);
}

public sealed class OrderWriteBatchHandler(IOrderStore store)
    : IRequestBatchHandler<OrderWriteRequest>
{
    public async ValueTask HandleAsync(
        IReadOnlyList<OrderWriteRequest> requests,
        CancellationToken cancellationToken = default)
    {
        await store.WriteBatchAsync(requests, cancellationToken);
    }
}

The handler's ValueTask is an internal completion signal that RequestBatcher awaits once. Application callers always receive Task.

Register the handler and choose its lifetime:

services.AddRequestBatcher<OrderWriteRequest, OrderWriteBatchHandler>(
    ServiceLifetime.Scoped,
    options =>
    {
        options.BatchSize = 256;
    });

Inject IRequestBatcher<TRequest> and submit requests:

public sealed class OrderService(IRequestBatcher<OrderWriteRequest> batcher)
{
    public Task SaveAsync(
        OrderWriteRequest request,
        CancellationToken cancellationToken = default) =>
        batcher.ProcessAsync(request, cancellationToken);
}

The returned Task completes after the handler has processed that request. If the handler fails, the caller receives the original exception.

If several handler calls fail for one explicit group, await follows normal Task semantics and throws one original exception. All distinct exception instances remain available through Task.Exception.InnerExceptions; one handler exception fanned out to several requests is recorded once.

When requests already exist as a group, submit them together:

await batcher.ProcessAsync(orderWriteRequests, cancellationToken);

This overload snapshots the group and submits it as one producer operation, then waits for every request. In Wait mode, an oversized group enters the queue as consecutive capacity-sized slices. In Fail mode, the whole group must fit immediately. The submission does not force one handler call; it may still be split by BatchSize or partition routing.

An explicit group is not a partition boundary. With more than one partition, every item is routed independently; the group is guaranteed to stay in one partition only when MaxConcurrency = 1, or when every item produces the same partition key.

Submission Behavior

Batching and Capacity

• A handler batch contains at most BatchSize requests that are already queued.
• RequestBatcher does not add a fixed delay or wait for a minimum batch size.
• MaxPendingRequests is the internal BufferQueue capacity shared by queued and currently handled requests. It does not limit the size of one explicit submission or the number of callers waiting for capacity.
• Wait admits a group no larger than the capacity atomically and splits a larger group into consecutive capacity-sized slices. Fail rejects the whole submission unless it fits immediately.
• Caller cancellation removes a request only before handler dispatch. After dispatch, the caller observes the real handler outcome.
• Shutdown rejects new calls and drains every submission that started before shutdown, including submissions waiting for capacity. Requests still cannot be recovered after a process failure.

Routing Modes

MaxConcurrency controls both handler concurrency and partition count:

One explicit group can therefore be handled concurrently by several partitions. Each handler invocation reads from one partition. Concurrent callers have no defined order before their requests are appended.

Partition routing is optional:

services.AddRequestBatcher<OrderWriteRequest, OrderWriteBatchHandler>(
    ServiceLifetime.Scoped,
    options =>
    {
        options.BatchSize = 256;
        options.MaxConcurrency = 4;

        // Optional. Without this selector, routing is round-robin.
        options.UsePartitionKey(request => request.OrderId);
    });

Equal finite, integer-valued numeric keys or equal non-null string keys are routed to one partition and processed there in append order. A partition key controls routing only: it does not force related requests into one handler batch or deduplicate them.

Partition-local ordering can support patterns such as merging repeated updates within each batch. Correctness across batches still requires application safeguards such as version checks, idempotency keys, unique constraints, or transactions.

Do not move an operation into RequestBatcher when it must commit or roll back with the caller's transaction.

The runnable PostgreSQL Web API sample shows batched upserts, version-protected updates, and deduplicated reads.

License

MIT