CosmoKv 3.2.3

CosmoKv

An embeddable, MVCC-transactional, LSM-tree key/value store for .NET 10. A C# port of Dgraph's Badger v4, redesigned for idiomatic .NET (ValueTask, IAsyncEnumerable<T>, ReadOnlySpan<byte>).

CosmoKv is a single-process, single-writer-many-reader store. Open one Db per directory; from there it's all async, lock-free reads, and snapshot-isolated transactions.

Features

• MVCC snapshot-isolation transactions. Every commit gets a monotonic timestamp; readers see a stable snapshot at their BeginTransaction call. Read-write conflicts are detected at commit time and raise ConflictException. A WaterMark prunes the oracle's committed-txn list so long-running processes don't grow it unbounded.
• WAL-backed durability. Every write goes through a write-ahead log before the in-memory memtable; SyncWrites=true fsyncs on every entry. Crash recovery is exercised by an in-process kill-fuzz test and a subprocess crash harness that asserts every SYNCED line is recoverable after Process.Kill.
• Value log indirection. Values larger than ValueThreshold (default 1 KiB) live in .vlog files; LSM rows hold a 12-byte pointer. Reclaim space via Db.RunValueLogGcAsync.
• Background compaction. A worker scores levels and runs L0→L1→…→L_n compactions; tombstones are discarded at the bottom level. L0 backpressure stalls writers cleanly when the compactor can't keep up. Old SSTable files are removed via FileShare.Delete so deletion works on POSIX and Windows even with open reader handles.
• Sharded LRU block cache (default 256 MiB, 16 shards) keyed by (fileId, blockOffset). Decodes once, serves many — cached point Gets are 3× faster than uncached.
• Pluggable block compression — None, LZ4 (K4os.Compression.LZ4), Snappy (IronSnappy), Zstd (ZstdSharp.Port). Codec id is stored per block so a single database can mix compression types across its history.
• TTL enforcement. Set expiresAt on a write and the entry stops being visible to GetAsync / IterateAsync after that wall-clock time. A background sweeper (default 60 s cadence, capped at 10 000 ops/cycle) writes tombstones for expired user-keys; Db.SweepExpiredAsync triggers a manual pass.
• Lock-free memtable. Writes use CAS-link inserts on a concurrent skiplist (Interlocked.CompareExchange on per-level Next pointers). Toggleable via DbOptions.UseLockFreeMemtable (default on).
• IAsyncEnumerable<Item> iteration with forward/reverse, prefix filter, and AllVersions mode. Lazy Item.ReadValueAsync defers vlog dereference until the consumer needs the bytes.
• Change-feed subscriptions. db.Subscribe(prefix) returns an IAsyncEnumerable<ChangeEvent> of every subsequent commit whose user-key matches the prefix. Each subscriber gets a bounded drop-oldest channel so slow consumers can't stall the writer.
• Bulk streaming. KvStream (export) + KvStreamWriter (bulk-load) ship IAsyncEnumerable<KvList> pipelines. The writer builds L0 SSTables directly, bypassing the WAL — measurably faster than per-key SetAsync for large fresh loads. Used by backup/restore.
• Backup / Restore. db.BackupAsync(stream) writes the live keyspace as a portable, CRC64-protected COSMOBAK stream; Db.RestoreAsync(targetOptions, stream) rehydrates a fresh database from one. Tamper + truncation are detected by the trailer CRC.
• Encryption at rest. Set DbOptions.EncryptionKey to a 32-byte master key and every SST block, WAL frame, and vlog frame is AEAD-encrypted with AES-256-GCM. A KEYREGISTRY in the db directory holds the wrapped DEK; the master key never touches disk. Tampered ciphertext fails the AEAD tag check; wrong master key fails to unwrap.
• OpenTelemetry instrumentation. Meter("CosmoKv") exposes write/read/commit/conflict/flush/compaction counters and latency histograms; ActivitySource("CosmoKv") emits spans for Set/Delete/Get/Commit/Flush/Compaction/VlogGc. Listen via dotnet-counters, dotnet-trace, or OTel AddMeter("CosmoKv").AddSource("CosmoKv"). Zero overhead when no listener is attached.
• Pooled buffers on the hot path. WalWriter, ValueLogFile, and TableBuilder rent their framing buffers from ArrayPool<byte>.Shared — large-value writes allocate up to 81% less than the naive implementation.

Quick example

using CosmoKv;

await using var db = await Db.OpenAsync(DbOptions.Default("/tmp/mydb"));

// Auto-commit single writes
await db.SetAsync("hello"u8.ToArray(), "world"u8.ToArray());
byte[]? v = await db.GetAsync("hello"u8.ToArray());                   // [ 'w','o','r','l','d' ]

// Multi-key atomic transactions
await using (var txn = db.BeginTransaction(update: true))
{
    txn.Set("a"u8.ToArray(), "1"u8.ToArray());
    txn.Set("b"u8.ToArray(), "2"u8.ToArray());
    await txn.CommitAsync();
}

// Sorted iteration with a prefix
await foreach (var item in db.IterateAsync(IteratorOptions.WithPrefix("user:"u8.ToArray())))
{
    byte[] value = await item.ReadValueAsync();
    // ...
}

Performance

Numbers from bench/CosmoKv.Benchmarks/ on Apple M1, APFS, SyncWrites=false, 1 warmup + 3 measurement iterations, against the v3.2 codebase (post-audit + Phase 6 fast-path, 2026-05-21). Reproduce with dotnet run -c Release --project bench/CosmoKv.Benchmarks -- --filter '*'.

v3.2 (Phase 6 Tier 1) adds an inline fast-path for single-call SetAsync / DeleteAsync that bypasses the writer-task channel when uncontended, recovering ~58 % of the per-call latency vs the pre-fast-path v3.1 measurements. Bulk writers still hit the channel path; WriteBatch and Txn remain the recommended route for high-throughput / high-fan-out workloads.

Cross-platform validation: 290 library + 14 stress + 11 CLI tests pass on both macOS Apple Silicon (APFS) and Windows 11 ARM64 (Parallels guest, native NTFS).

See docs/benchmarks.md for the full tables across hosts and the per-compression breakdown.

Installation

CosmoKv is published to nuget.org:

dotnet add package CosmoKv

There's also a CLI:

dotnet tool install -g CosmoKv.Cli
cosmokv --help

Or as a project reference if you've cloned the repo:

<ItemGroup>
  <ProjectReference Include="path/to/CosmoKv/src/CosmoKv/CosmoKv.csproj" />
</ItemGroup>

Requires .NET 10 (LangVersion latest, Nullable enabled). CosmoKv depends on System.IO.Hashing 9.0 plus the block-compression codecs K4os.Compression.LZ4, IronSnappy, and ZstdSharp.Port.

Documentation

• docs/getting-started.md — first-time user guide with worked examples.
• docs/examples.md — end-to-end snippets: counters, transfers, bulk load, secondary indexes, snapshot reports, time-travel reads, vlog GC.
• docs/api-reference.md — every public type and method.
• docs/benchmarks.md — full benchmark tables across macOS + Windows-NTFS, plus the pool vs. ValueTask attribution.
• docs/internals.md — architecture, on-disk formats, MVCC mechanics, tuning, and a note on what's intentionally not implemented.

Building from source

git clone <repo>
cd CosmoKv
dotnet build                                        # builds the library, tests, and bench
dotnet test --filter "Category!=Stress"             # fast loop, ~15 s
dotnet test --filter "Category=Stress"              # bank + fuzz, ~30 s default
dotnet run -c Release --project bench/CosmoKv.Benchmarks -- --filter '*'   # BenchmarkDotNet

Test coverage: 228 library tests + 11 CLI tests + ~10 stress tests, all green on macOS (Apple Silicon, .NET 10) and Windows 11 ARM64 (Parallels guest, .NET 10.0.300).

Status

3.2.0 — deep-dive audit + single-call write fast-path. Three audit phases between v2.2 and v3.1 closed P0/P1 correctness bugs (vlog GC race, snapshot-isolation smear, encrypted-backup plaintext leak, encryption AAD relocation attack, FD leak under steady compaction), and Phase 6 then recovered the per-call SetAsync latency regression those phases inadvertently amplified — current single-call throughput is roughly 2× faster than v3.1 and within measurement noise of macOS APFS / Windows-NTFS / Linux ext4 of each other. Cross-platform tests stay at 290 library + 14 stress + 11 CLI.

⚠ v3.0 wire-format break — encrypted databases written by v2.x are NOT readable by v3.x (WAL/vlog AEAD AAD changed from fileId to (fileId, frameOffset) to defeat intra-file frame-relocation attacks). Reseed from backup if upgrading from a pre-v3 encrypted DB.

See CHANGELOG.md for the full version-by-version log.

Out of scope (and likely to stay that way):

• On-disk format compatibility with Badger files (CosmoKv has its own wire format).
• Multi-process access to one directory (no file lock; still single-process).

See docs/internals.md for the full deferred-features list and rationale.

Releasing

NuGet publish is automated by .github/workflows/publish.yml.

To cut a release:

git tag v3.2.0
git push origin v3.2.0

The workflow restores, builds in Release, runs the non-stress test suite, packs both CosmoKv.<version>.nupkg (library) and CosmoKv.Cli.<version>.nupkg (CLI tool) with the tag's version (the leading v is stripped), pushes them to nuget.org, and creates a GitHub Release with auto-generated notes and the packages attached.

For a manual run without tagging, dispatch the workflow from the Actions tab and pass the version as an input.

Required repository secret: NUGET_API_KEY — an API key from nuget.org/account/apikeys scoped to push the CosmoKv package.

The build is reproducible (ContinuousIntegrationBuild=true under GitHub Actions) and Source Link is embedded, so consumers can step-into-source on debug.

License

Apache License 2.0. CosmoKv is a port of the original Badger, which is also Apache 2.0.