Caching 4.0.0

Caching

<img src="https://github.com/jchristn/Caching/raw/master/assets/icon.png" height="128" width="128">

High-performance, thread-safe caching library for .NET with FIFO and LRU eviction policies, automatic expiration, persistence support, and comprehensive event notifications.

What Is This Library?

Caching is a lightweight, production-ready caching library that provides:

• FIFO (First-In-First-Out) Cache: Evicts the oldest entries when capacity is reached
• LRU (Least Recently Used) Cache: Evicts the least recently accessed entries
• Thread-Safe: All operations are fully thread-safe for concurrent access
• Automatic Expiration: Time-based expiration with sliding or absolute TTL
• Event Notifications: Comprehensive events for cache operations
• Persistence Layer: Optional persistence to disk or custom storage
• Statistics Tracking: Built-in hit/miss rates, eviction counts, and performance metrics
• Memory Limits: Optional memory-based eviction in addition to count-based
• Modern API: GetOrAdd, AddOrUpdate, and async-ready patterns

Installation

dotnet add package Caching

Or via Package Manager:

Install-Package Caching

Quick Start

Basic FIFO Cache

using Caching;

// Create a FIFO cache with capacity of 1000, evicting 100 items when full
var cache = new FIFOCache<string, Person>(capacity: 1000, evictCount: 100);

// Add items
cache.AddReplace("user:123", new Person { Name = "Alice", Age = 30 });

// Get items
Person person = cache.Get("user:123");

// Try pattern (no exceptions)
if (cache.TryGet("user:123", out Person p))
{
    Console.WriteLine($"Found: {p.Name}");
}

// Remove items
cache.Remove("user:123");

// Dispose when done
cache.Dispose();

Basic LRU Cache

// LRU evicts least recently accessed items
var cache = new LRUCache<string, byte[]>(capacity: 500, evictCount: 50);

cache.AddReplace("image:1", imageBytes);
cache.Get("image:1"); // Updates last-used timestamp

cache.Dispose();

Key Features

1. Expiration

Absolute Expiration

// Expires at specific time
cache.AddReplace("session:xyz", sessionData, DateTime.UtcNow.AddMinutes(30));

// Or use TimeSpan for relative expiration
cache.AddReplace("temp:data", tempData, TimeSpan.FromSeconds(60));

Sliding Expiration

// Enable sliding expiration (TTL refreshes on access)
cache.SlidingExpiration = true;

cache.AddReplace("sliding:key", value, TimeSpan.FromMinutes(5));
// Each time you access the item, expiration resets to 5 minutes from now
cache.Get("sliding:key"); // Refreshes expiration

2. GetOrAdd Pattern

// Atomically get existing or create new value
var person = cache.GetOrAdd("user:456", key =>
{
    // This factory only runs if key doesn't exist
    return database.GetPerson(456);
});

// With expiration
var data = cache.GetOrAdd("data:789",
    key => LoadExpensiveData(key),
    TimeSpan.FromHours(1));

3. AddOrUpdate Pattern

// Add if new, update if exists
var result = cache.AddOrUpdate(
    "counter:visits",
    addValue: 1,
    updateValueFactory: (key, oldValue) => oldValue + 1);

Console.WriteLine($"Visit count: {result}");

4. Events

cache.Events.Added += (sender, e) => Console.WriteLine($"Added: {e.Key}");

cache.Events.Replaced += (sender, e) => Console.WriteLine($"Replaced: {e.Key}");

cache.Events.Removed += (sender, e) => Console.WriteLine($"Removed: {e.Key}");

cache.Events.Evicted += (sender, keys) => Console.WriteLine($"Evicted {keys.Count} items");

cache.Events.Expired += (sender, key) => Console.WriteLine($"Expired: {key}");

cache.Events.Cleared += (sender, e) => Console.WriteLine("Cache cleared");

cache.Events.Disposed += (sender, e) => Console.WriteLine("Cache disposed");

5. Persistence

Implement the IPersistenceDriver<TKey, TValue> interface:

public class FilePersistence : IPersistenceDriver<string, string>
{
    private readonly string _directory;

    public FilePersistence(string directory)
    {
        _directory = directory;
        Directory.CreateDirectory(directory);
    }

    public void Write(string key, string data)
    {
        File.WriteAllText(Path.Combine(_directory, key), data);
    }

    public string Get(string key)
    {
        return File.ReadAllText(Path.Combine(_directory, key));
    }

    public void Delete(string key)
    {
        File.Delete(Path.Combine(_directory, key));
    }

    public void Clear()
    {
        foreach (var file in Directory.GetFiles(_directory))
            File.Delete(file);
    }

    public bool Exists(string key)
    {
        return File.Exists(Path.Combine(_directory, key));
    }

    public List<string> Enumerate()
    {
        return Directory.GetFiles(_directory)
            .Select(Path.GetFileName)
            .ToList();
    }
}

// Use with cache
var persistence = new FilePersistence("./cache_data");
var cache = new LRUCache<string, string>(1000, 100, persistence);

// Restore from persistence on startup
cache.Prepopulate();

// All add/remove operations automatically persist
cache.AddReplace("key", "value"); // Written to disk
cache.Remove("key");              // Deleted from disk

6. Statistics

var cache = new FIFOCache<string, object>(1000, 100);

// Perform operations
cache.AddReplace("key1", "value1");
cache.Get("key1");        // Hit
cache.TryGet("missing", out _); // Miss

// Get statistics
var stats = cache.GetStatistics();

Console.WriteLine($"Hit Rate: {stats.HitRate:P}");
Console.WriteLine($"Hits: {stats.HitCount}");
Console.WriteLine($"Misses: {stats.MissCount}");
Console.WriteLine($"Evictions: {stats.EvictionCount}");
Console.WriteLine($"Expirations: {stats.ExpirationCount}");
Console.WriteLine($"Current Count: {stats.CurrentCount}");
Console.WriteLine($"Capacity: {stats.Capacity}");

// Reset counters
cache.ResetStatistics();

7. Memory Limits

var cache = new FIFOCache<string, byte[]>(10000, 100);

// Limit cache to 100MB
cache.MaxMemoryBytes = 100 * 1024 * 1024;

// Provide size estimator for your value type
cache.SizeEstimator = bytes => bytes.Length;

// Cache will evict entries if memory limit is exceeded
cache.AddReplace("large", new byte[10 * 1024 * 1024]); // 10MB

Console.WriteLine($"Memory used: {cache.CurrentMemoryBytes} bytes");

8. Configuration Options

var cache = new LRUCache<int, string>(1000, 100);

// Sliding expiration
cache.SlidingExpiration = true;

// Expiration check interval (default: 1000ms)
cache.ExpirationIntervalMs = 500;

// Memory limits
cache.MaxMemoryBytes = 50 * 1024 * 1024; // 50MB
cache.SizeEstimator = str => str.Length * 2; // Unicode estimation

API Reference

Core Methods

Properties

Thread Safety

All cache operations are thread-safe and can be called concurrently from multiple threads:

var cache = new LRUCache<int, string>(10000, 100);

// Safe to call from multiple threads
Parallel.For(0, 1000, i =>
{
    cache.AddReplace(i, $"value{i}");
    cache.TryGet(i, out _);
    if (i % 10 == 0) cache.Remove(i);
});

Performance Tips

1. Choose the Right Cache Type:

   • Use FIFO when access patterns don't matter (e.g., time-series data)
   • Use LRU when recent items are more likely to be accessed again

2. Set Appropriate Capacity:

   • Monitor HitRate to tune capacity
   • Higher capacity = better hit rate but more memory

3. Tune EvictCount:

   • Larger EvictCount = fewer eviction operations but more items removed at once
   • Smaller EvictCount = more frequent evictions but finer-grained

4. Use TryGet for Optional Lookups:

   • TryGet is faster than catching exceptions from Get

5. Minimize Event Handler Work:

   • Events fire synchronously; keep handlers fast
   • Offload heavy work to background tasks

6. Memory Limits:

   • Only use MaxMemoryBytes if needed; it adds overhead
   • Provide accurate SizeEstimator for best results

Migrating from v3.x

Most code will work unchanged. Key changes:

// v3.x
cache.Events.Added = handler; // Overwrites all handlers! ❌

// v4.0
cache.Events.Added += handler; // Adds handler ✅

// v3.x (abstract class)
public class MyDriver : IPersistenceDriver<string, string>
{
    public override void Write(string key, string data) { }
}

// v4.0 (interface)
public class MyDriver : IPersistenceDriver<string, string>
{
    public void Write(string key, string data) { } // No 'override'
}

Contributing

Contributions are welcome! Please open an issue or PR on GitHub.

License

See LICENSE.md