feels.Dank.Cache.LRU 1.0.0

LRU Cache for .NET

A high-performance, thread-safe LRU (Least Recently Used) cache implementation for .NET with TTL support, usage statistics, and asynchronous operations.

Features

• ✅ Thread-safe - Full concurrency support for all operations
• ⚡ O(1) Complexity - Constant time operations for core functionality
• 🕒 Time-To-Live (TTL) - Automatic expiration of cache items with Absolute and Sliding modes
• 📊 Usage Statistics - Track hits, misses, evictions, failures, and timeouts
• 🔄 Async Support - First-class asynchronous operations with cancellation tokens
• 🧪 Testable Design - Interface-based implementation for easy mocking and testing
• 🧹 Automatic Cleanup - Background task for removing expired items
• 🔒 Production Ready - Battle-tested implementation with circuit breaker and deadlock prevention
• 📈 Performance Metrics - Built-in telemetry with System.Diagnostics.Metrics
• ⏱️ Timeout Protection - Prevent hanging value factories with configurable timeouts
• 🌐 High Concurrency - Shard-based locking for optimal parallelism

Installation

Install via NuGet Package Manager:

Install-Package feels.Dank.Cache.LRU

Or via .NET CLI:

dotnet add package feels.Dank.Cache.LRU

Quick Start

Basic Usage

// Create a cache with capacity of 100 items, 5-minute TTL, and sliding expiration
var cache = new LruCache<string, int>(
    capacity: 100, 
    defaultTtl: TimeSpan.FromMinutes(5),
    expirationMode: ExpirationMode.Sliding
);

// Get or add value (synchronous)
var value = cache.GetOrAdd("key", k => ComputeValue(k));

// Get or add value with timeout (prevents hanging operations)
var valueWithTimeout = cache.GetOrAdd("key", k => ComputeValue(k), TimeSpan.FromSeconds(2));

// Get or add value (asynchronous)
var asyncValue = await cache.GetOrAddAsync("key", async (k, ct) => await ComputeValueAsync(k, ct));

// Manual operations
cache.AddOrUpdate("key", 42, TimeSpan.FromMinutes(10)); // Custom TTL for this item
cache.TryGet("key", out var result);
cache.TryGetWithMetadata("key", out var value, out var ttl, out var created, out var accessed);
cache.Invalidate("key");

Dependency Injection

Register with your favorite DI container:

// In your service registration
services.AddSingleton<ILruCache<string, int>>(_ => 
    new LruCache<string, int>(
        capacity: 100, 
        defaultTtl: TimeSpan.FromMinutes(5),
        expirationMode: ExpirationMode.Sliding,
        cleanupInterval: TimeSpan.FromMinutes(2)
    ));

API Reference

Constructor

public LruCache(
    int capacity, 
    TimeSpan? defaultTtl = null,
    ExpirationMode expirationMode = ExpirationMode.Absolute,
    TimeSpan? cleanupInterval = null,
    TimeSpan? timeCacheInterval = null
)

• capacity: Maximum number of items in the cache (must be > 0)
• defaultTtl: Optional default time-to-live for cache items
• expirationMode: Absolute (from creation) or Sliding (from last access) expiration
• cleanupInterval: Interval for background cleanup of expired items
• timeCacheInterval: Interval for caching current time (optimization)

Properties

Core Methods

GetOrAdd(TKey key, Func<TKey, TValue> valueFactory, TimeSpan? timeout = null)

Gets the value associated with the key, or adds a new value using the factory.

• Thread Safety: Guarantees single execution of valueFactory per key
• Statistics: Increments Hits/Misses counters accordingly
• LRU: Updates item position in the LRU list
• Timeout: Optional timeout to prevent hanging operations
• Circuit Breaker: Automatically blocks requests after repeated failures
• Deadlock Prevention: Throws exception on recursive cache access

GetOrAddAsync(TKey key, Func<TKey, CancellationToken, Task<TValue>> valueFactory, TimeSpan? timeout = null, CancellationToken cancellationToken = default)

Asynchronous version of GetOrAdd with cancellation support.

• Concurrency: All threads for the same key wait for first operation to complete
• Cancellation: Supports operation cancellation via token
• Timeout: Optional timeout to prevent hanging operations
• Memory Leak Prevention: Proper cleanup on cancellation

TryGet(TKey key, out TValue value)

Tries to get a value from the cache.

• Returns true if item was found and valid
• Updates item position in LRU list
• Increments appropriate statistics counter

TryGetWithMetadata(TKey key, out TValue value, out TimeSpan? remainingTtl, out DateTime creationTime, out DateTime lastAccessTime)

Tries to get a value with metadata.

• Returns TTL information and access timestamps
• Useful for monitoring and diagnostics

AddOrUpdate(TKey key, TValue value, TimeSpan? ttl = null, ExpirationMode? expirationMode = null)

Adds or updates an item in the cache.

• Resets TTL timer for existing items
• Supports custom TTL and expiration mode per item
• Moves item to front of LRU list
• Handles capacity constraints automatically

Expiration Management

ClearOlderThan(TimeSpan olderThan)

Removes items that:

• Haven't been accessed for longer than specified duration
• Have TTL exceeding specified duration

RemoveExpiredItems()

Removes all items that have exceeded their TTL.

Refresh(TKey key)

Manually updates an item's access time (resets TTL timer).

ExtendTtl(TKey key, TimeSpan? newTtl = null)

Extends the TTL of an item without changing its value.

Cache Maintenance

Examples

Basic Caching Pattern

public class UserService
{
    private readonly ILruCache<int, User> _userCache;
    
    public UserService()
    {
        _userCache = new LruCache<int, User>(
            capacity: 500,
            defaultTtl: TimeSpan.FromMinutes(10),
            expirationMode: ExpirationMode.Sliding
        );
    }
    
    public User GetUser(int userId)
    {
        return _userCache.GetOrAdd(userId, id => 
            Database.GetUser(id) ?? throw new UserNotFoundException(id)
        );
    }
    
    public User GetUserWithTimeout(int userId)
    {
        try {
            return _userCache.GetOrAdd(userId, id => Database.GetUser(id), TimeSpan.FromSeconds(3));
        }
        catch (TimeoutException ex) {
            // Handle timeout
            return FallbackUser;
        }
    }
    
    public async Task<User> GetUserAsync(int userId, CancellationToken ct = default)
    {
        return await _userCache.GetOrAddAsync(userId, async (id, token) => 
            await Database.GetUserAsync(id, token), TimeSpan.FromSeconds(5), ct);
    }
}

Monitoring Cache Performance

var cache = new LruCache<string, object>(1000);

// Perform cache operations...

// Check performance metrics
Console.WriteLine($"Cache Hit Rate: {cache.Hits / (double)(cache.Hits + cache.Misses):P2}");
Console.WriteLine($"Failure Rate: {cache.Failures / (double)(cache.Hits + cache.Misses):P2}");
Console.WriteLine($"Total Items: {cache.Count}");
Console.WriteLine($"Evictions: {cache.Evictions}");
Console.WriteLine($"Circuit Breaker Activations: {cache.CircuitBreakerActivations}");

// Get detailed metadata for an item
if (cache.TryGetWithMetadata("important-key", out var value, out var ttl, out var created, out var accessed))
{
    Console.WriteLine($"Remaining TTL: {ttl?.TotalSeconds} seconds");
    Console.WriteLine($"Created: {created}, Last Accessed: {accessed}");
}

Handling Failures and Timeouts

var cache = new LruCache<string, string>(
    capacity: 100,
    defaultTtl: TimeSpan.FromMinutes(5)
);

try 
{
    // This will throw if valueFactory exceeds 2 seconds
    var result = cache.GetOrAdd("key", k => 
    {
        // Simulate slow operation
        Thread.Sleep(3000);
        return "value";
    }, TimeSpan.FromSeconds(2));
}
catch (TimeoutException ex)
{
    Console.WriteLine("Operation timed out - using fallback");
    // Use fallback value
}

try
{
    // This will activate circuit breaker after repeated failures
    for (int i = 0; i < 10; i++)
    {
        try {
            cache.GetOrAdd("failing-key", k => throw new InvalidOperationException("Simulated failure"));
        }
        catch {
            // Ignore
        }
    }
    
    // This call will fail immediately due to circuit breaker
    cache.GetOrAdd("failing-key", k => "This won't execute");
}
catch (InvalidOperationException ex)
{
    Console.WriteLine($"Circuit breaker active: {ex.Message}");
    // Wait and retry later
}

Performance Characteristics

Memory Considerations

• Each cache item has a small overhead (approximately 70-120 bytes)
• The cache will never exceed the specified capacity
• TTL adds minimal overhead (two additional DateTime fields per item)
• Background cleanup runs in a separate thread with configurable interval

Concurrency Performance

• Shard-based locking (16 shards by default) significantly reduces contention
• Time caching reduces calls to DateTime.UtcNow
• Async-friendly design prevents thread pool starvation
• Timeout protection prevents single slow operations from blocking the cache

Best Practices

1. Choose appropriate capacity: Monitor Evictions counter to determine if capacity is too low
2. Select proper expiration mode: Use Sliding for frequently accessed items, Absolute for time-bound data
3. Set meaningful TTL: Prevent stale data while maintaining good hit rate
4. Use timeouts: Always specify timeouts for valueFactory operations
5. Monitor statistics: Track Failures and CircuitBreakerActivations for problem detection
6. Use async methods for I/O-bound operations to avoid blocking threads
7. Pre-warm the cache during application startup for critical paths
8. Use metadata methods for diagnostics and monitoring
9. Configure cleanup interval based on your TTL settings
10. Handle circuit breaker exceptions with appropriate fallback strategies

Advanced Configuration

Background Cleanup

By default, expired items are cleaned up every minute. Adjust this based on your TTL:

// Cleanup every 30 seconds for short TTLs
var cache = new LruCache<string, object>(1000, 
    defaultTtl: TimeSpan.FromSeconds(30),
    cleanupInterval: TimeSpan.FromSeconds(15)
);

Time Caching

For high-throughput scenarios, increase time caching interval:

// Cache time for 50ms instead of default 10ms
var cache = new LruCache<string, object>(1000,
    timeCacheInterval: TimeSpan.FromMilliseconds(50)
);

Sliding vs Absolute Expiration

// Session cache - sliding expiration (reset timer on each access)
var sessionCache = new LruCache<string, Session>(1000, 
    TimeSpan.FromMinutes(30), 
    ExpirationMode.Sliding
);

// News feed cache - absolute expiration (fixed duration from creation)
var newsCache = new LruCache<string, News>(500, 
    TimeSpan.FromMinutes(5), 
    ExpirationMode.Absolute
);

Troubleshooting

Common Issues and Solutions

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests. When contributing:

1. Follow the existing code style and documentation standards
2. Include unit tests for new functionality
3. Update documentation as needed
4. Ensure all existing tests pass

License

This project is licensed under the MIT License - see the LICENSE file for details.