CosmoKv 2.2.1

CosmoKv

An embeddable, MVCC-transactional, LSM-tree key/value store for .NET 10. A C# port of Dgraph's Badger v4, redesigned for idiomatic .NET (ValueTask, IAsyncEnumerable<T>, ReadOnlySpan<byte>).

CosmoKv is a single-process, single-writer-many-reader store. Open one Db per directory; from there it's all async, lock-free reads, and snapshot-isolated transactions.

Features

• MVCC snapshot-isolation transactions. Every commit gets a monotonic timestamp; readers see a stable snapshot at their BeginTransaction call. Read-write conflicts are detected at commit time and raise ConflictException. A WaterMark prunes the oracle's committed-txn list so long-running processes don't grow it unbounded.
• WAL-backed durability. Every write goes through a write-ahead log before the in-memory memtable; SyncWrites=true fsyncs on every entry. Crash recovery is exercised by an in-process kill-fuzz test and a subprocess crash harness that asserts every SYNCED line is recoverable after Process.Kill.
• Value log indirection. Values larger than ValueThreshold (default 1 KiB) live in .vlog files; LSM rows hold a 12-byte pointer. Reclaim space via Db.RunValueLogGcAsync.
• Background compaction. A worker scores levels and runs L0→L1→…→L_n compactions; tombstones are discarded at the bottom level. L0 backpressure stalls writers cleanly when the compactor can't keep up. Old SSTable files are removed via FileShare.Delete so deletion works on POSIX and Windows even with open reader handles.
• Sharded LRU block cache (default 256 MiB, 16 shards) keyed by (fileId, blockOffset). Decodes once, serves many — cached point Gets are 3× faster than uncached.
• Pluggable block compression — None, LZ4 (K4os.Compression.LZ4), Snappy (IronSnappy), Zstd (ZstdSharp.Port). Codec id is stored per block so a single database can mix compression types across its history.
• TTL enforcement. Set expiresAt on a write and the entry stops being visible to GetAsync / IterateAsync after that wall-clock time. A background sweeper (default 60 s cadence, capped at 10 000 ops/cycle) writes tombstones for expired user-keys; Db.SweepExpiredAsync triggers a manual pass.
• Lock-free memtable. Writes use CAS-link inserts on a concurrent skiplist (Interlocked.CompareExchange on per-level Next pointers). Toggleable via DbOptions.UseLockFreeMemtable (default on).
• IAsyncEnumerable<Item> iteration with forward/reverse, prefix filter, and AllVersions mode. Lazy Item.ReadValueAsync defers vlog dereference until the consumer needs the bytes.
• Change-feed subscriptions. db.Subscribe(prefix) returns an IAsyncEnumerable<ChangeEvent> of every subsequent commit whose user-key matches the prefix. Each subscriber gets a bounded drop-oldest channel so slow consumers can't stall the writer.
• Bulk streaming. KvStream (export) + KvStreamWriter (bulk-load) ship IAsyncEnumerable<KvList> pipelines. The writer builds L0 SSTables directly, bypassing the WAL — measurably faster than per-key SetAsync for large fresh loads. Used by backup/restore.
• Backup / Restore. db.BackupAsync(stream) writes the live keyspace as a portable, CRC64-protected COSMOBAK stream; Db.RestoreAsync(targetOptions, stream) rehydrates a fresh database from one. Tamper + truncation are detected by the trailer CRC.
• Encryption at rest. Set DbOptions.EncryptionKey to a 32-byte master key and every SST block, WAL frame, and vlog frame is AEAD-encrypted with AES-256-GCM. A KEYREGISTRY in the db directory holds the wrapped DEK; the master key never touches disk. Tampered ciphertext fails the AEAD tag check; wrong master key fails to unwrap.
• OpenTelemetry instrumentation. Meter("CosmoKv") exposes write/read/commit/conflict/flush/compaction counters and latency histograms; ActivitySource("CosmoKv") emits spans for Set/Delete/Get/Commit/Flush/Compaction/VlogGc. Listen via dotnet-counters, dotnet-trace, or OTel AddMeter("CosmoKv").AddSource("CosmoKv"). Zero overhead when no listener is attached.
• Pooled buffers on the hot path. WalWriter, ValueLogFile, and TableBuilder rent their framing buffers from ArrayPool<byte>.Shared — large-value writes allocate up to 81% less than the naive implementation.

Quick example

using CosmoKv;

await using var db = await Db.OpenAsync(DbOptions.Default("/tmp/mydb"));

// Auto-commit single writes
await db.SetAsync("hello"u8.ToArray(), "world"u8.ToArray());
byte[]? v = await db.GetAsync("hello"u8.ToArray());                   // [ 'w','o','r','l','d' ]

// Multi-key atomic transactions
await using (var txn = db.BeginTransaction(update: true))
{
    txn.Set("a"u8.ToArray(), "1"u8.ToArray());
    txn.Set("b"u8.ToArray(), "2"u8.ToArray());
    await txn.CommitAsync();
}

// Sorted iteration with a prefix
await foreach (var item in db.IterateAsync(IteratorOptions.WithPrefix("user:"u8.ToArray())))
{
    byte[] value = await item.ReadValueAsync();
    // ...
}

Performance

Numbers from bench/CosmoKv.Benchmarks/ on Apple M1, APFS, SyncWrites=false, 1 warmup + 3 measurement iterations. Reproduce with dotnet run -c Release --project bench/CosmoKv.Benchmarks -- --filter '*'.

Allocations on the write path are pool-backed: a 10 000 × 1024-byte random write batch allocates just 2.5 MB, down from 13.4 MB in the unpooled baseline (-81 %).

Cross-platform validation: 228 library + 11 CLI tests (239 total) pass on macOS Apple Silicon; 228 library tests pass on Windows 11 ARM64 (Parallels guest, native NTFS).

See docs/benchmarks.md for the full tables across hosts and a breakdown of how pooling + ValueTask each contributed.

Installation

CosmoKv is published to nuget.org:

dotnet add package CosmoKv

There's also a CLI:

dotnet tool install -g CosmoKv.Cli
cosmokv --help

Or as a project reference if you've cloned the repo:

<ItemGroup>
  <ProjectReference Include="path/to/CosmoKv/src/CosmoKv/CosmoKv.csproj" />
</ItemGroup>

Requires .NET 10 (LangVersion latest, Nullable enabled). CosmoKv depends on System.IO.Hashing 9.0 plus the block-compression codecs K4os.Compression.LZ4, IronSnappy, and ZstdSharp.Port.

Documentation

• docs/getting-started.md — first-time user guide with worked examples.
• docs/examples.md — end-to-end snippets: counters, transfers, bulk load, secondary indexes, snapshot reports, time-travel reads, vlog GC.
• docs/api-reference.md — every public type and method.
• docs/benchmarks.md — full benchmark tables across macOS + Windows-NTFS, plus the pool vs. ValueTask attribution.
• docs/internals.md — architecture, on-disk formats, MVCC mechanics, tuning, and a note on what's intentionally not implemented.

Building from source

git clone <repo>
cd CosmoKv
dotnet build                                        # builds the library, tests, and bench
dotnet test --filter "Category!=Stress"             # fast loop, ~15 s
dotnet test --filter "Category=Stress"              # bank + fuzz, ~30 s default
dotnet run -c Release --project bench/CosmoKv.Benchmarks -- --filter '*'   # BenchmarkDotNet

Test coverage: 228 library tests + 11 CLI tests + ~10 stress tests, all green on macOS (Apple Silicon, .NET 10) and Windows 11 ARM64 (Parallels guest, .NET 10.0.300).

Status

2.1.0 — every v1.0 deferred item shipped. Phases 17–23 closed out the rest of the deferred-features list: ReadOnly enforcement, multi-worker compactor, txn iter reads-own-writes (v1.1), write-path lock split (v1.2), change-feed subscriptions + bulk stream/writer (v1.3), backup/restore (v1.4), AES-256-GCM encryption at rest with KeyRegistry (v2.0), and the cosmokv CLI (v2.1). See CHANGELOG.md for the full version-by-version log.

Out of scope (and likely to stay that way):

• On-disk format compatibility with Badger files (CosmoKv has its own wire format).
• Multi-process access to one directory (no file lock; still single-process).

See docs/internals.md for the full deferred-features list and rationale.

Releasing

NuGet publish is automated by .github/workflows/publish.yml.

To cut a release:

git tag v2.1.0
git push origin v2.1.0

The workflow restores, builds in Release, runs the non-stress test suite, packs both CosmoKv.<version>.nupkg (library) and CosmoKv.Cli.<version>.nupkg (CLI tool) with the tag's version (the leading v is stripped), pushes them to nuget.org, and creates a GitHub Release with auto-generated notes and the packages attached.

For a manual run without tagging, dispatch the workflow from the Actions tab and pass the version as an input.

Required repository secret: NUGET_API_KEY — an API key from nuget.org/account/apikeys scoped to push the CosmoKv package.

The build is reproducible (ContinuousIntegrationBuild=true under GitHub Actions) and Source Link is embedded, so consumers can step-into-source on debug.

License

Apache License 2.0. CosmoKv is a port of the original Badger, which is also Apache 2.0.