CSharpDB.Storage 1.8.0

CSharpDB.Storage

CSharpDB.Storage is the page-oriented durability layer used by the CSharpDB embedded database engine. It owns:

• physical file I/O through IStorageDevice
• page caching and dirty tracking through Pager
• write-ahead logging and crash recovery through WriteAheadLog
• row-id keyed B+trees for table and index storage
• schema metadata persistence through SchemaCatalog

This package is usually consumed indirectly through CSharpDB.Engine, but it also supports direct low-level use for tooling, diagnostics, and storage experiments.

Most users: configure storage through Database

If you are using SQL or the engine layer, customize storage like this:

using CSharpDB.Engine;

var options = new DatabaseOptions()
    .ConfigureStorageEngine(builder =>
    {
        builder.UseLookupOptimizedPreset();
    });

await using var db = await Database.OpenAsync("app.cdb", options);

UseLookupOptimizedPreset() is the current recommended opt-in preset for file-backed lookup-heavy workloads. It sets MaxCachedPages = 2048 and keeps the standard B-tree index provider, which outperformed the caching index wrapper in the current tuning matrix.

For sustained durable writes, use the write-heavy preset instead:

using CSharpDB.Engine;

var options = new DatabaseOptions()
    .ConfigureStorageEngine(builder =>
    {
        builder.UseWriteOptimizedPreset();
    });

await using var db = await Database.OpenAsync("ingest.cdb", options);

UseWriteOptimizedPreset() is the current recommended opt-in preset for file-backed write-heavy workloads. It keeps the existing cache and index configuration, raises the auto-checkpoint frame threshold to 4096, and runs auto-checkpoints in background slices instead of blocking the triggering commit. PagerOptions.AutoCheckpointMaxPagesPerStep controls how much work each background slice performs; the default remains 64 pages. In the current durable-write diagnostics, the 64-page background preset was the best measured background variant at 33.30K ops/sec, slightly ahead of 256 pages at 33.24K and foreground FrameCount(4096) at 33.13K. The important difference is that background sliced mode kept checkpoint work off essentially all commits while the foreground policy still had 246 commits pay checkpoint cost in the median run.

Low-level use: open the storage graph directly

If you need direct access to Pager, SchemaCatalog, or BTree, use the default storage engine factory:

using CSharpDB.Storage.BTrees;
using CSharpDB.Storage.Paging;
using CSharpDB.Storage.StorageEngine;

var storageOptions = new StorageEngineOptionsBuilder()
    .UsePagerOptions(new PagerOptions { MaxCachedPages = 1024 })
    .UseBTreeIndexes()
    .Build();

var factory = new DefaultStorageEngineFactory();
var context = await factory.OpenAsync("lowlevel.cdb", storageOptions);
await using var pager = context.Pager;

await pager.BeginTransactionAsync();
try
{
    uint rootPageId = await BTree.CreateNewAsync(pager);
    var tree = new BTree(pager, rootPageId);

    await tree.InsertAsync(1, new byte[] { 1, 2, 3, 4 });
    byte[]? payload = await tree.FindAsync(1);

    await pager.CommitAsync();
}
catch
{
    await pager.RollbackAsync();
    throw;
}

Key extension points

• IStorageDevice for alternate storage backends
• IPageCache through PagerOptions.PageCacheFactory
• ICheckpointPolicy for auto-checkpoint decisions
• IPageOperationInterceptor for diagnostics and fault injection
• IPageChecksumProvider for WAL checksum behavior
• IIndexProvider for index-store composition
• ISerializerProvider for record and schema serialization
• ICatalogStore for catalog payload encoding
• IStorageEngineFactory for replacing the default storage composition root

Related docs

• Storage tutorial index
• Storage architecture
• Usage and extensibility guide
• Runnable study examples

Related packages

Installation

dotnet add package CSharpDB.Storage

For the all-in-one package:

dotnet add package CSharpDB