TursoSync 1.9.2

TursoSync

The offline-sync + full-text-search layer for Turso on .NET. It adds local↔cloud replication (push/pull/checkpoint against the Turso sync engine) and tantivy full-text search — the pieces the official Turso.Data.Sqlite binding doesn't ship. It carries a familiar ADO.NET surface (modeled on Turso.Data) so Dapper, DbUp and the DbProviderFactory pattern work, but in its own Turso.Sync namespace so it coexists with the official package rather than colliding with it.

Reach for the official Turso.Data.Sqlite for base local/remote access, the SQLite-compat facade, EF Core, and NativeAOT. Reach for TursoSync when you need offline replication or FTS — the two are complementary.

using Turso.Sync;

// Local-only (offline fast path — plain engine, no sync overhead)
await using var conn = new TursoConnection("Data Source=app.db");
await conn.OpenAsync();

// …or synced against Turso Cloud
await using var synced = new TursoConnection(
    "Data Source=app.db;Remote Url=libsql://my-db.turso.io;Auth Token=…");
await synced.OpenAsync();
synced.SyncDatabase!.Push();   // push local changes
synced.SyncDatabase!.Pull();   // pull + apply remote changes

Packages

dotnet add package TursoSync
dotnet add package TursoSync.DbUp     # optional: migrations
dotnet add package TursoSync.Dapper   # optional: Dapper type handlers

Highlights

• Two lanes, one API. No Remote Url → the plain local engine (AsyncIO=0, no IO pump); a remote (or Sync=true) → the sync engine. In release builds, local performance is on par with SQLite.
• Familiar surface, own namespace. Public types live in namespace Turso.Sync (TursoConnection, TursoCommand, …), modeled on Turso.Data so Dapper, DbUp and the DbProviderFactory pattern work — and namespaced so a project can reference both this and the official Turso.Data.Sqlite without clashing.
• Connection pooling on by default (Pooling=false to disable) — opening Turso is expensive, the pool makes the open-per-op pattern ~50× cheaper.
• Extensibility: CreateFunction / CreateAggregate / CreateCollation / EnableExtensions / LoadExtension.
• Local at-rest encryption: …;Encryption Cipher=aes256gcm;Encryption Key=<hex>.
• Full-text search (tantivy): …;Experimental Index Method=true enables CREATE INDEX … USING fts. Composes with encryption — the index lives inside the database file, so it's encrypted at rest too.

Connection string keys

Data Source (required) · Remote Url · Auth Token · Namespace · Bootstrap · Sync · Pooling · Busy Timeout · Long Poll Timeout · Http Timeout · Push Operations Threshold · Encryption Cipher · Encryption Key · Experimental Index Method.

Push batching

Pushes default to batches of 1,000 CDC operations, so a backlog accumulated offline can make progress without uploading the entire change set in one request. Set Push Operations Threshold in the connection string, or TursoSyncConfig.PushOperationsThreshold, to change the threshold. Zero explicitly restores one request for the whole change set. A batch ends at a transaction boundary, so one larger transaction can exceed the threshold. A retry resumes from the last batch the server acknowledged.

Connections to the same replica share one sync engine and must agree on the threshold. A conflicting threshold is rejected when opening a connection. Close the existing connections and call TursoConnection.ClearPool() before reopening with a different value.

Sync HTTP diagnostics

The opt-in HTTP diagnostics API requires TursoSync 1.8.0 or later. Automatic gzip response decompression is available from 1.9.2. Applications must upgrade their TursoSync package reference to use these features.

Set TursoSyncConfig.HttpDiagnostics before TursoSyncDatabase.Create to include bootstrap requests, or use TursoSyncDatabase.HttpDiagnostics on an open replica to log a push:

var database = synced.SyncDatabase!;
var previous = database.HttpDiagnostics;
try
{
    database.HttpDiagnostics = diagnostic => Console.WriteLine(diagnostic);
    database.Push();
}
finally
{
    database.HttpDiagnostics = previous;
}

The callback defaults to null (off). It receives one TursoSyncHttpDiagnostic when an HTTP request finishes or fails: method, an allowlisted endpoint label (info, pull-updates, push-updates, or other), request body bytes, time to response headers, status, declared response bytes, actual bytes read, total duration, and FailurePhase (None, BeforeHeaders, or ResponseBody). Unknown methods become OTHER. No URL, query, arbitrary path, headers, credentials, payload, or exception text is included. Null measurements mean unavailable; partial body bytes survive a read failure. HTTP error statuses are reported unchanged and are not themselves transport failures. Existing sync error handling and timeouts are unchanged.

The transport negotiates gzip and decompresses responses as they arrive. HTTP diagnostics count compressed response bytes; the native TursoStats.NetworkReceivedBytes counter counts the decoded payload. Read idle timeouts still apply to network reads below decompression.

Callbacks run synchronously under the sync operation lock; keep them fast and do not call back into the database. Callback exceptions are swallowed. Timings exclude the callback. Events arrive on completion, not while a request is stalled; an infinite HTTP timeout can therefore postpone an event indefinitely. The callback is shared by all connections using the same replica and captured per request: coordinate temporary changes with other database users if logging only one push.

Engine fixes and large backlogs

The native build applies the maintained patches under patches/ to the exact turso-engine.json revision. Both release and integration builds include those patches, and their cache keys include the patch contents. local-pack.sh applies them before building from source; when supplying TURSOSYNC_NATIVE_DIR, supply a native built with the same patches.

Pull/replay retains the original local CDC transaction boundaries inside the atomic apply transaction. A later push can therefore keep using bounded batches; real transactions larger than the threshold still remain intact. A backlog already collapsed by an earlier engine cannot regain its lost boundaries. Set Http Timeout=-1 to allow its large upload to finish without a client deadline.

The pinned 0.8.3 engine also includes the upstream CDC upsert fix. It captures the previous row before an ON CONFLICT DO UPDATE overwrites it. Reading that row afterward could produce an incorrect before-image or fail with invalid text/blob errors on sync replicas.

Full-image replay updates rows with stable, non-null primary keys in place, retaining child references. Push also combines an adjacent DELETE/INSERT pair for the same key within one source transaction into an upsert, so replacement pairs already queued by older replayers can finish uploading. This normalization applies to the standard push path without a custom row transform. It never combines changes across source transactions, and the acknowledgment uses the original insert's CDC change ID. Foreign-key checks remain enabled.

Incremental pulls compare incoming pages with the restored remote database image before replaying local changes. Byte-identical pages are skipped when appending to the WAL, apart from the final frame needed to record the commit and database size. Replace-base snapshots and partial replicas retain their existing apply path. This reduces local WAL growth when the cloud returns whole storage segments; it does not reduce which pages the server sends.

Full-text search

Tantivy FTS is behind the engine's experimental index_method feature, so opt in with Experimental Index Method=true (or TursoSyncConfig.ExperimentalIndexMethod). Without it, USING fts fails to parse.

await using var conn = new TursoConnection(
    "Data Source=app.db;Experimental Index Method=true");
await conn.OpenAsync();

conn.ExecuteNonQuery("CREATE INDEX idx_articles ON articles USING fts (title, body)");

// BM25-ranked search, with the matched terms highlighted.
using var cmd = conn.CreateCommand();
cmd.CommandText = """
    SELECT id, fts_highlight(body, '<mark>', '</mark>', @q)
    FROM articles
    WHERE fts_match(title, body, @q)
    ORDER BY fts_score(title, body, @q) DESC
    """;

Tokenizers (default/raw/simple/whitespace/ngram) and per-column weights are set with WITH (tokenizer = 'ngram', weights = 'title=2.0,body=1.0'). Automatic segment merging is disabled — run OPTIMIZE INDEX <name> on a cadence (like VACUUM/ANALYZE) after bulk or continuous inserts.

Native library

The provider P/Invokes the turso_sync_sdk_kit native (the Turso sync engine + tantivy FTS). Released packages carry it under runtimes/<rid>/native/. For local development against a self-built engine, point TURSOSYNC_NATIVE_DIR at the folder containing the built turso_sync_sdk_kit library. Build the native in release — debug builds are ~25× slower.

Engine version

The Turso engine (the turso_sync_sdk_kit native) is not vendored — CI builds it from tursodatabase/turso at the commit pinned in turso-engine.json, currently the stable v0.8.3 release. The pin keeps builds reproducible and lets us validate each engine bump before adopting it.

• Bump it with scripts/bump-turso.sh <tag|latest> (resolves the tag → commit SHA).
• A weekly Engine bump workflow opens a PR when a newer release appears in the pinned series; CI builds
  • tests against it on the PR, so an ABI change is caught in review before merge.

Examples

// Migrations (TursoSync.DbUp)
using DbUp;
var result = DeployChanges.To.TursoDatabase("Data Source=app.db")
    .WithScriptsEmbeddedInAssembly(typeof(Program).Assembly)
    .Build()
    .PerformUpgrade();

// Dapper type handlers (TursoSync.Dapper)
Turso.TursoTypeHandlers.Register();

// A scalar UDF
conn.CreateFunction("times_two", 1, args => Convert.ToInt64(args[0]) * 2);

Status

TursoSync is the sync + FTS complement to the official Turso.Data.Sqlite binding, not a replacement for it. Base local/remote access, the SQLite-compat facade, EF Core and NativeAOT are official's domain — use that package directly for those. TursoSync owns the offline sync engine (push/pull/checkpoint) and tantivy FTS, and carries just enough of an ADO.NET surface (modeled on Turso.Data, see TURSO-PARITY.md) to query a synced database and support Dapper/DbUp. One known gap is sync-lane at-rest encryption (base-lane encryption is supported).

Testing

rig test (or dotnet test) runs the unit suite. The live sync suites are gated on environment variables and report Inconclusive (skip) when unset:

• LiveSyncIntegrationTests — needs TURSOSYNC_TEST_SYNC_SERVER pointing at a tursodb binary; the harness starts a tursodb --sync-server on a free port per test.
• TursoSyncBehaviorTests — needs TURSOSYNC_TEST_REMOTE_URL (+ TURSOSYNC_TEST_REMOTE_TOKEN) for a real Turso Cloud round-trip.

Copy .env.example to .env at the repo root and fill in what you have — the test project loads it automatically. Real environment variables (and CI, which exports these) always take precedence.

Releasing

Releases are driven by shipRig (changesets) — config in .changeset/.

1. Record intent: shiprig add (pick the bump + summary; the three packages are fixed, so they version together; TursoSync.Tests is ignored).
2. Ship: shiprig release → versions + changelog → commit → tags TursoSync@x.y.z → push.

The pushed tag triggers the Release workflow, which builds all six RID natives (release + FTS, stripped), packs, and publishes to NuGet via trusted publishing (OIDC — no API key). shipRig only versions/tags/pushes; the cross-arch native build + publish stay in CI (.changeset/release.jsonc).

Never run shiprig publish here. It isn't in the pipeline, and .changeset/config.json names no package source, so a local publish would pack on your machine, without the six native builds CI makes, and push that to nuget.org. The only reason it doesn't today is that the version on disk is already published, so the push is skipped as a duplicate. After a local shiprig version it would ship a broken package.

Day-to-day dev uses rig (.rig.json): rig build / rig test / rig coverage, plus rig engine <tag|latest> (pin the engine) and rig pack (local Tier-0 pack + consume test).

License

MIT.