Microlens.Cache 2.0.0

Microlens.Cache

Stampede-protected, container-scoped in-process caching for .NET, built on Microsoft.Extensions.Caching.Memory.

Microlens.Cache is a high-performance caching library that deduplicates, stores, refreshes and evicts cached values through a single, thread-safe ICachingService.

Unlike using IMemoryCache directly, Microlens.Cache guarantees that a factory runs once per key no matter how many callers miss at the same time, keeps serving the previous value while a refresh loads, and enforces per-container eviction policies (LRU or LFU) with their own capacity, expiration and configuration.

Why Microlens.Cache? | Quick Start | Quick Example | Features | Configuration | API Reference | Behavior & Guarantees | Performance Characteristics | Comparison | Supported Frameworks | When Not To Use Microlens.Cache | License

Why Microlens.Cache?

Caching looks simple until it runs under concurrency. Most hand-rolled and IMemoryCache-based caches eventually hit the same problems:

• Cache Stampede: many concurrent misses run the same expensive factory at the same time.
• Refresh Gaps: evicting before reloading leaves a window where every caller misses.
• Unbounded Growth: entries without expiration or capacity limits grow until memory pressure hits.
• Stale Writes: a slow load started before an invalidation overwrites the newer state after it.
• Mixed Workloads: one global cache cannot give hot lookups and bulk reference data different policies.

Examples of typical use cases:

• Database Lookups: Cache entities by key without flooding the database on cold start.
• Reference Data: Load an entire collection (countries, currencies, settings) once and serve lookups from it.
• External API Responses: Deduplicate concurrent calls to rate-limited or slow third-party services.
• Configuration & Feature Data: Refresh periodically while callers keep reading the current value.
• Hot-Key Workloads: Keep the most frequently used entries with LFU eviction under a fixed capacity.

Quick Start

Installation

dotnet add package Microlens.Cache

Register Services

builder.Services.AddMicrolensCache();

Inject and Use

public sealed class ProductService(ICachingService cache, IProductRepository repository) {
    public Task<Product?> GetAsync(string id, CancellationToken cancellationToken) {
        return cache.GetOrAddAsync("products", id, () => repository.FindAsync(id), cancellationToken: cancellationToken);
    }
}

That's it.

Concurrent misses for the same key share a single factory execution.

Quick Example

Given 1,000 concurrent requests for the same product on a cold cache:

var product = await cache.GetOrAddAsync(
    "products",
    "sku-42",
    () => repository.FindAsync("sku-42"),
    CacheExpiration.AfterWrite(TimeSpan.FromMinutes(5)));

Microlens.Cache executes the factory once, and all 1,000 callers receive the same result.

Request 1    ─┐
Request 2    ─┤
Request 3    ─┼──► factory() ──► Product sku-42 ──► all callers
...          ─┤
Request 1000 ─┘

• No locks to write.
• No Lazy<T> wrappers to manage.
• No duplicate database round-trips.
• No partial state on failure.
• No stale writes after invalidation.

Features

Microlens.Cache is NOT a distributed cache. It is an in-process cache that focuses on correctness under concurrency, predictable eviction and low overhead on the read path.

Stampede Protection

Every miss is coordinated through a single in-flight entry per key.

• The first caller runs the factory.
• Every concurrent caller awaits the same result.
• A failed or cancelled factory is not cached; all waiters receive the same exception, and the next call retries.
• null results are cached like any other value.

Containers

Entries are grouped into named containers, each backed by its own MemoryCache with its own eviction policy, capacity and default expiration.

await cache.GetOrAddAsync("users", userId, () => LoadUserAsync(userId));
await cache.GetOrAddAsync("settings", "theme", () => LoadThemeAsync());

Collections

Load an entire dictionary once and serve key lookups from it.

CacheResult<Country> result = await cache.GetOrAddAsync<string, Country>("catalog", "countries", "PK", LoadCountriesAsync);

if (result.Found) {
    Console.WriteLine(result.Value);
}

• CacheResult<TValue>.Found distinguishes an absent key from a key holding default.
• An absent key reloads the collection at most once per AbsentKeyRebuildInterval (default 30 seconds), so lookups of keys that do not exist can never flood the source.
• Ownership of the returned dictionary transfers to the cache; a FrozenDictionary gives the fastest lookups on .NET 8+.

Stale-While-Revalidate Refresh

Force a fresh value without creating a miss window.

await cache.GetOrAddAsync("settings", "theme", () => LoadThemeAsync(), refresh: true);

• Concurrent readers keep receiving the previous value until the new one completes.
• Concurrent refreshes of the same key coalesce into one factory execution.
• A failed refresh leaves the previous value in place.

Expiration

CacheExpiration.AfterWrite(TimeSpan.FromMinutes(10));                           // absolute
CacheExpiration.AfterAccess(TimeSpan.FromMinutes(2));                           // sliding
new CacheExpiration(TimeSpan.FromHours(1), TimeSpan.FromMinutes(5));            // both

• Lifetime starts when the value is stored, never while its factory is still running.
• A per-call expiration overrides the container default, which overrides the global default.

Eviction Policies

• Every scalar value or collection counts as one entry toward Capacity.
• Each pass evicts down to Capacity − Capacity × CompactionPercentage.
• LFU frequencies are halved on every pass, so formerly hot entries cannot pin the container forever.
• A refreshed LFU key keeps its frequency.

Explicit Writes & Invalidation

cache.Set("settings", "theme", theme, CacheExpiration.AfterWrite(TimeSpan.FromHours(1)));
cache.SetCollection("catalog", "countries", countries);

cache.Remove("settings", "theme");
cache.RemoveCollection("catalog", "countries");
cache.Clear("settings");

• Set, Remove and Clear supersede any load already in flight; data fetched before them is never written back.
• TryGet reads completed values only and never triggers a load.

Cancellation

await cache.GetOrAddAsync("products", id, () => LoadAsync(id), cancellationToken: cancellationToken);

A CancellationToken cancels the caller's wait only. The shared factory runs to completion and still populates the cache for every other caller.

Safety Checks

Misuse fails loudly at the call site instead of hanging or corrupting state:

• Reading a key as a different type than it was stored with throws InvalidOperationException.
• A factory that awaits or refreshes its own in-flight key throws InvalidOperationException instead of deadlocking.
• null arguments throw ArgumentNullException synchronously.
• Invalid configuration throws InvalidOperationException when the service is constructed.

Configuration

Configure a global default and per-container overrides through CacheOptions.

builder.Services.AddMicrolensCache(options => {
    options.Defaults.Expiration = CacheExpiration.AfterWrite(TimeSpan.FromMinutes(30));

    var users = options.Container("users");
    users.Eviction = Registry.EvictionPolicy.Lfu;
    users.Capacity = 10_000;

    var sessions = options.Container("sessions");
    sessions.Eviction = Registry.EvictionPolicy.Lru;
    sessions.Capacity = 50_000;
    sessions.Expiration = CacheExpiration.AfterAccess(TimeSpan.FromMinutes(20));
});

ContainerOptions

• Containers without their own entry use Defaults.
• Container, key and collection names are case-sensitive (ordinal).
• Options are copied when a container is first used; later changes do not affect live containers.
• AddMicrolensCache can be called multiple times; configuration actions compose and the service is registered once.

API Reference

All members are exposed through ICachingService, registered as a singleton.

Behavior & Guarantees

Performance Characteristics

Microlens.Cache is designed for hot read paths and high-concurrency workloads.

Key implementation details:

• Lock-free cache hits through a single MemoryCache.TryGetValue
• Allocation-free scalar hits: the stored completed Task<TValue> is returned as-is
• Atomic single-flight coordination through ConcurrentDictionary add, swap and compare-remove
• Striped per-key locks guarding writes, invalidations and evictions only, never reads
• System.Threading.Lock on .NET 9+ targets
• Continuations run asynchronously, so completing a load never executes waiter code on the factory's thread
• ConfigureAwait(false) throughout, safe for synchronous callers on .NET Framework
• Interned collection keys and cached delegates, so steady-state lookups allocate no keys or closures
• LFU frequency tracking without interlocked operations on the hit path

The library is optimized for throughput and predictable latency while preserving strict correctness under concurrency.

Comparison

Microlens.Cache builds on IMemoryCache for storage and expiration, and adds the coordination, refresh and eviction semantics it lacks.

Supported Frameworks

Dependencies:

• Microsoft.Extensions.Caching.Memory
• Microsoft.Extensions.DependencyInjection.Abstractions
• Microsoft.Extensions.Options

When Not To Use Microlens.Cache

Microlens.Cache is not intended for:

• Distributed or shared caching across multiple processes or servers
• Persisting cached data across application restarts
• Replacing Redis, IDistributedCache or HybridCache in multi-instance deployments
• Caching values that must be evicted by exact memory size in bytes

If your data must be shared between instances or survive restarts, use a distributed cache.

License

Licensed under the Apache License 2.0.