AutoCache 4.0.0

AutoCache

Predictable stale-while-revalidate caching and per-key single-flight coordination for high-throughput .NET applications.

AutoCache wraps the storage backend you already use. It prevents a cold cache miss from becoming a source stampede, returns stale data without waiting while one refresh runs, and stops serving data at its hard-expiration boundary.

• One source call for a concurrent cold miss or stale refresh
• Immediate stale responses during background revalidation
• Caller cancellation that does not abort work shared by other callers
• Source deadlines, refresh-failure backoff, and optional refresh jitter
• Removable per-key flight state with bounded outage tracking
• System.Diagnostics.Metrics and ActivitySource instrumentation
• No runtime NuGet dependencies
• Source Link metadata and a NuGet symbol package for debugging
• .NET 8 and .NET 10 support

AutoCache coordinates work inside one adapter instance and one process. Storage, serialization, distributed locking, and cache topology stay under application control.

Install

Install the stable Version 4 package:

dotnet add package AutoCache --version 4.0.0

How it works

Each stored value is wrapped in an envelope containing RefreshAt and ExpiresAt timestamps. Given a one-minute fresh period and one-hour expiration:

A failed, timed-out, or empty background refresh leaves the stale value intact until hard expiration. Failed refreshes are temporarily backed off so an unhealthy dependency is not retried for every stale request.

Quick start

1. Implement storage

Adapters only implement persistence. This example is deliberately small; a production adapter should serialize values, enforce its own I/O deadlines, and apply the supplied hard TTL.

using System.Collections.Concurrent;
using AutoCache;

public sealed class ProductCache : CacheAdapter
{
    private sealed record Stored(object Value, DateTimeOffset ExpiresAt);
    private readonly ConcurrentDictionary<string, Stored> _entries = new();

    public ProductCache()
        : base(new AutoCacheOptions
        {
            FreshFor = TimeSpan.FromMinutes(2),
            ExpireAfter = TimeSpan.FromHours(1),
            FactoryTimeout = TimeSpan.FromSeconds(10),
            RefreshFailureBackoff = TimeSpan.FromSeconds(5)
        }) { }

    public override Task SetAsync<T>(string key, T value, TimeSpan hardTtl)
    {
        _entries[key] = new Stored(value!, DateTimeOffset.UtcNow + hardTtl);
        return Task.CompletedTask;
    }

    public override Task<(T Value, bool Found)> GetAsync<T>(string key)
    {
        if (!_entries.TryGetValue(key, out var entry) || DateTimeOffset.UtcNow >= entry.ExpiresAt)
            return Task.FromResult((default(T)!, false));

        return Task.FromResult(((T)entry.Value, true));
    }

    public override Task RemoveAsync(string key)
    {
        _entries.TryRemove(key, out _);
        return Task.CompletedTask;
    }
}

Register one shared, thread-safe adapter instance so all requests participate in the same flights:

builder.Services.AddSingleton<ProductCache>();

2. Load through AutoCache

var product = await cache.GetOrCreateAsync(
    $"product:{productId}",
    async cancellationToken =>
    {
        var value = await repository.FindAsync(productId, cancellationToken);

        return value is null
            ? AutoCacheFactoryResult<Product>.NoValue()
            : AutoCacheFactoryResult<Product>.FromValue(value);
    },
    new AutoCachePolicy
    {
        FreshFor = TimeSpan.FromMinutes(1),
        ExpireAfter = TimeSpan.FromHours(1),
        FactoryTimeout = TimeSpan.FromSeconds(3),
        RefreshFailureBackoff = TimeSpan.FromSeconds(5)
    },
    requestCancellationToken);

NoValue() on a cold miss raises AutoCacheSourceUnavailableException because no usable value exists. The same result during a background refresh preserves and returns the stale value.

Policy

Configure defaults on the adapter and override individual values with AutoCachePolicy when needed.

The source receives the factory-timeout token. A request token only cancels that caller's wait; it does not cancel a flight that other requests may need. A source that ignores cancellation can continue after the caller receives a timeout, but its late result is not cached and its eventual fault is observed.

Factory deadlines do not include cache-backend reads or writes. Apply appropriate timeouts in the adapter itself.

Observability

Subscribe to the AutoCache meter and activity source with OpenTelemetry or System.Diagnostics listeners.

Stable source names are also exposed through AutoCacheDiagnostics.

3.x migration

The storage interface and original GetOrCreateAsync calling shape remain available for gradual migration:

var value = await cache.GetOrCreateAsync(
    "todo-service",
    async () =>
    {
        var result = await service.GetAsync();
        return (result, HasValue: true);
    },
    refreshAt: TimeSpan.FromMinutes(2),
    expireAt: TimeSpan.FromHours(1),
    timeout: TimeSpan.FromSeconds(30));

Version 4 stores a richer envelope and is not wire-compatible with 3.x entries. Use a new key prefix/version or clear old entries during rollout. See MIGRATION.md for all behavior changes.

Validation

The release suite covers cold, stale, and expired stampedes; exact expiration boundaries; source and storage failures; cancellation; synchronous factory timeouts; late results; JSON roundtrips; bounded backoff cleanup; and repeated outage/recovery cycles.

./scripts/verify-release.ps1 `
  -TestRepeats 5 `
  -Requests 1000000 `
  -Concurrency 4096 `
  -Keys 10000 `
  -Rounds 100

The script performs a clean warnings-as-errors build, tests .NET 8 and .NET 10, runs synchronous and delayed/serialized load scenarios, packs the NuGet artifact, and restores an independent consumer from it. See RELEASE-VALIDATION.md for recorded results.

These are library-level and synthetic-backend tests. Validate the actual adapter, source dependencies, serialized data, and expected traffic in staging before production rollout.

Operational boundaries

• Single-flight state belongs to one adapter instance and is not shared across processes.
• Use a consistent value type and policy per key; coalesced callers share the winning factory and policy.
• A factory may outlive the request that supplied it. Do not capture dependencies disposed with that request.
• RemoveAsync does not cancel an active factory, which can later repopulate the key.
• At the failure-backoff cap, additional failing keys are not tracked and may retry with subsequent traffic.
• AutoCache is not a distributed lock, circuit breaker, serializer, or L1/L2 cache framework.

Contributing and releases

Run ./scripts/verify-release.ps1 before a pull request. GitHub Actions repeats release verification on Windows and Linux. NuGet publication runs only through the release workflow after its requested version matches the project, using NuGet trusted publishing with a short-lived credential.

See CHANGELOG.md for release history.