Microlens.Cache 2.0.2

Microlens.Cache

Stampede-protected, container-scoped in-process caching for .NET, built on Microsoft.Extensions.Caching.Memory.

Microlens.Cache is a high-performance caching library that deduplicates, stores, refreshes and evicts cached values through a single, thread-safe ICachingService.

Unlike using IMemoryCache directly, Microlens.Cache guarantees that a factory runs once per key no matter how many callers miss at the same time, keeps serving the previous value while a refresh loads, and enforces per-container eviction policies (LRU or LFU) with their own capacity, expiration and configuration.

What's New | Why Microlens.Cache? | Requirements | Quick Start | Quick Example | Features | Configuration | Migrating from 1.x | API Reference | Behavior & Guarantees | Metrics | Performance Characteristics | Limitations | Comparison | When Not To Use Microlens.Cache | License

What's New

2.0 is a major release with breaking changes. See Migrating from 1.x.

• CacheRegistry: EvictionPolicy moved from Registry to CacheRegistry. Registry is now internal.
• CacheExpiration.Never: Replaces CacheExpiration.Default, which meant "never expires", not "use the container default". CacheExpiration is now sealed.
• Cancellable factories: New state-passing overloads give the factory a CancellationToken that is cancelled when the load is superseded by Set, Remove, RemoveCollection or Clear, or when the service is disposed.
• Allocation-free calls: The state-passing overloads avoid the per-call closure.
• Metrics: System.Diagnostics.Metrics counters on the Microlens.Cache meter.
• Fixed: Concurrent refresh: true calls now coalesce into one factory execution.
• Fixed: A load can no longer stay pinned as in-flight if storing its result throws.
• Fixed: RemoveCollection releases the collection's interned key.
• Faster hits: None and LRU containers no longer write to the entry on a cache hit.

Why Microlens.Cache?

Caching looks simple until it runs under concurrency. Most hand-rolled and IMemoryCache-based caches eventually hit the same problems:

• Cache Stampede: many concurrent misses run the same expensive factory at the same time.
• Refresh Gaps: evicting before reloading leaves a window where every caller misses.
• Unbounded Growth: entries without expiration or capacity limits grow until memory pressure hits.
• Stale Writes: a slow load started before an invalidation overwrites the newer state after it.
• Mixed Workloads: one global cache cannot give hot lookups and bulk reference data different policies.

Examples of typical use cases:

• Database Lookups: Cache entities by key without flooding the database on cold start.
• Reference Data: Load an entire collection (countries, currencies, settings) once and serve lookups from it.
• External API Responses: Deduplicate concurrent calls to rate-limited or slow third-party services.
• Configuration & Feature Data: Refresh periodically while callers keep reading the current value.
• Hot-Key Workloads: Keep the most frequently used entries with LFU eviction under a fixed capacity.

Requirements

Dependencies:

• Microsoft.Extensions.Caching.Memory
• Microsoft.Extensions.DependencyInjection.Abstractions
• Microsoft.Extensions.Options
• System.Diagnostics.DiagnosticSource (.NET Framework / .NET Standard 2.0 only)

Quick Start

Installation

dotnet add package Microlens.Cache

Register Services

using Microlens.Cache.Extensions;

builder.Services.AddMicrolensCache();

Inject and Use

using Microlens.Cache.Services;

public sealed class ProductService(ICachingService cache, IProductRepository repository) {
    public Task<Product?> GetAsync(string id, CancellationToken cancellationToken) {
        return cache.GetOrAddAsync("products", id, () => repository.FindAsync(id), cancellationToken: cancellationToken);
    }
}

That's it.

Concurrent misses for the same key share a single factory execution.

Quick Example

Given 1,000 concurrent requests for the same product on a cold cache:

var product = await cache.GetOrAddAsync(
    "products",
    "sku-42",
    () => repository.FindAsync("sku-42"),
    CacheExpiration.AfterWrite(TimeSpan.FromMinutes(5)));

Microlens.Cache executes the factory once, and all 1,000 callers receive the same result.

Request 1    ─┐
Request 2    ─┤
Request 3    ─┼──► factory() ──► Product sku-42 ──► all callers
...          ─┤
Request 1000 ─┘

• No locks to write.
• No Lazy<T> wrappers to manage.
• No duplicate database round-trips.
• No partial state on failure.
• No stale writes after invalidation.

Features

Microlens.Cache is NOT a distributed cache. It is an in-process cache that focuses on correctness under concurrency, predictable eviction and low overhead on the read path.

Stampede Protection

Every miss is coordinated through a single in-flight entry per key.

• The first caller runs the factory.
• Every concurrent caller awaits the same result.
• A failed or cancelled factory is not cached; all waiters receive the same exception, and the next call retries.
• null results are cached like any other value.

Containers

Entries are grouped into named containers, each backed by its own MemoryCache with its own eviction policy, capacity and default expiration.

await cache.GetOrAddAsync("users", userId, () => LoadUserAsync(userId));
await cache.GetOrAddAsync("settings", "theme", () => LoadThemeAsync());

Collections

Load an entire dictionary once and serve key lookups from it.

CacheResult<Country> result = await cache.GetOrAddAsync<string, Country>("catalog", "countries", "PK", LoadCountriesAsync);

if (result.Found) {
    Console.WriteLine(result.Value);
}

• CacheResult<TValue>.Found distinguishes an absent key from a key holding default.
• An absent key reloads the collection at most once per AbsentKeyRebuildInterval (default 30 seconds), so lookups of keys that do not exist can never flood the source.
• Ownership of the returned dictionary transfers to the cache; a FrozenDictionary gives the fastest lookups on .NET 8+.

State-Passing & Cancellable Factories

The overloads in Microlens.Cache.Extensions pass state to the factory instead of capturing it, and give the factory a CancellationToken:

using Microlens.Cache.Extensions;

var product = await cache.GetOrAddAsync(
    "products",
    id,
    (repository, id),
    static (state, token) => state.repository.FindAsync(state.id, token));

• A static lambda with explicit state allocates no closure per call.
• The factory's token is cancelled when Set, Remove, RemoveCollection or Clear supersedes the load, or when the service is disposed. Callers awaiting that load observe the factory's outcome (OperationCanceledException if it honors the token).
• The collection variant, GetOrAddAsync<TState, TKey, TValue>(container, collection, key, state, factory, …), behaves the same way.

Stale-While-Revalidate Refresh

Force a fresh value without creating a miss window.

await cache.GetOrAddAsync("settings", "theme", () => LoadThemeAsync(), refresh: true);

• The refreshing caller waits for a value loaded after the one it saw; other callers keep receiving the previous value.
• Concurrent refreshes of the same key coalesce into one factory execution.
• A refresh with nothing cached behaves like a normal miss and joins any load already in flight.
• A failed refresh leaves the previous value in place.

Expiration

CacheExpiration.AfterWrite(TimeSpan.FromMinutes(10));                   // absolute
CacheExpiration.AfterAccess(TimeSpan.FromMinutes(2));                   // sliding
new CacheExpiration(TimeSpan.FromHours(1), TimeSpan.FromMinutes(5));    // both
CacheExpiration.Never;                                                  // never expires

• Lifetime starts when the value is stored, never while its factory is still running.
• A per-call expiration overrides the container default, which overrides the global default. Pass null to use the default.

Eviction Policies

• Every scalar value or collection counts as one entry toward Capacity.
• Each pass evicts down to Capacity − Capacity × CompactionPercentage.
• LFU frequencies are halved on every pass, so formerly hot entries cannot pin the container forever.
• A refreshed LFU key keeps its frequency.

Explicit Writes & Invalidation

cache.Set("settings", "theme", theme, CacheExpiration.AfterWrite(TimeSpan.FromHours(1)));
cache.SetCollection("catalog", "countries", countries);

cache.Remove("settings", "theme");
cache.RemoveCollection("catalog", "countries");
cache.Clear("settings");

• Set, Remove and Clear supersede any load already in flight; data fetched before them is never written back.
• TryGet reads completed values only and never triggers a load.

Cancellation

await cache.GetOrAddAsync("products", id, () => LoadAsync(id), cancellationToken: cancellationToken);

A CancellationToken passed by the caller cancels the caller's wait only. The shared factory keeps running for every other caller.

• Func<Task<TValue>> factories are never cancelled; a superseded load runs to completion and its result is returned to its callers but never stored.
• State-passing factories receive a token cancelled on supersession or disposal (see State-Passing & Cancellable Factories).

Safety Checks

Misuse fails loudly at the call site instead of hanging or corrupting state:

• Reading a key as a different type than it was stored with throws InvalidOperationException.
• A factory that awaits or refreshes its own in-flight key throws InvalidOperationException instead of deadlocking.
• null arguments throw ArgumentNullException synchronously.
• Invalid configuration throws InvalidOperationException when the service is constructed.

Configuration

Configure a global default and per-container overrides through CacheOptions.

using Microlens.Cache.Contracts;
using Microlens.Cache.Extensions;
using Microlens.Cache.Shared;

builder.Services.AddMicrolensCache(options => {
    options.Defaults.Expiration = CacheExpiration.AfterWrite(TimeSpan.FromMinutes(30));

    var users = options.Container("users");
    users.Eviction = CacheRegistry.EvictionPolicy.Lfu;
    users.Capacity = 10_000;

    var sessions = options.Container("sessions");
    sessions.Eviction = CacheRegistry.EvictionPolicy.Lru;
    sessions.Capacity = 50_000;
    sessions.Expiration = CacheExpiration.AfterAccess(TimeSpan.FromMinutes(20));
});

ContainerOptions

• Containers without their own entry use Defaults.
• Container, key and collection names are case-sensitive (ordinal).
• Options are copied when a container is first used; later changes do not affect live containers.
• AddMicrolensCache can be called multiple times; configuration actions compose and the service is registered once.

Migrating from 1.x

The value of every EvictionPolicy member is unchanged, so configuration bindings keep working.

API Reference

ICachingService (Microlens.Cache.Services) is registered as a singleton.

Extensions (Microlens.Cache.Extensions):

On a custom ICachingService implementation, the extensions fall back to the Func<Task<TValue>> overloads, with a token that is never cancelled.

Behavior & Guarantees

Metrics

Meter: Microlens.Cache. Every instrument is tagged with container.

Nothing is recorded unless a listener (OpenTelemetry, dotnet-counters) subscribes to the meter.

Performance Characteristics

Microlens.Cache is designed for hot read paths and high-concurrency workloads.

Key implementation details:

• Lock-free cache hits through a single MemoryCache.TryGetValue
• Allocation-free scalar hits: the stored completed Task<TValue> is returned as-is
• No write to the entry on a hit unless the container uses LFU
• Atomic single-flight coordination through ConcurrentDictionary add, swap and compare-remove
• Striped per-key locks guarding writes, invalidations and evictions only, never reads
• System.Threading.Lock on .NET 9+ targets
• Continuations run asynchronously, so completing a load never executes waiter code on the factory's thread
• ConfigureAwait(false) throughout, safe for synchronous callers on .NET Framework
• Interned collection keys, so steady-state collection lookups allocate no keys
• State-passing overloads resolved through struct loaders, so hits allocate no closures
• LFU frequency tracking without interlocked operations on the hit path
• Metric instruments checked for listeners before recording

The library is optimized for throughput and predictable latency while preserving strict correctness under concurrency.

Limitations

• In-process only: Values are not shared across processes and do not survive restarts.
• Entry-count capacity: Capacity counts entries, not bytes; a collection counts as one entry regardless of its size.
• LRU at capacity: New entries are rejected, not cached, until native background compaction frees space.
• Func<Task<TValue>> factories: They cannot be cancelled; use the state-passing overloads for cooperative cancellation.
• Metrics scope: Only GetOrAdd calls are counted; TryGet is not.

Comparison

Microlens.Cache builds on IMemoryCache for storage and expiration, and adds the coordination, refresh and eviction semantics it lacks.

When Not To Use Microlens.Cache

Microlens.Cache is not intended for:

• Distributed or shared caching across multiple processes or servers
• Persisting cached data across application restarts
• Replacing Redis, IDistributedCache or HybridCache in multi-instance deployments
• Caching values that must be evicted by exact memory size in bytes

If your data must be shared between instances or survive restarts, use a distributed cache.

License

Licensed under the Apache License 2.0.