FlatWire 1.1.0

FlatWire (.NET)

Streaming JSON serialization that keeps memory flat and time linear. Stream large collections element-by-element instead of materializing the whole payload, so peak memory is bounded by the largest single element — not the collection size. Built on System.Text.Json; wire format is plain JSON.

Part of the cross-language flatwire project (identical API in Python, Node, .NET, Rust, Go, and Java).

Install

dotnet add package FlatWire

Usage

using FlatWire;

// Whole value
byte[] bytes = Flat.Encode(new { hello = "world" });
var value = Flat.Decode<MyType>(bytes);

// Stream a large collection — flat memory
Flat.EncodeArray(items, stream);
await foreach (var row in Flat.DecodeArray<Row>(stream))
{
    // one element at a time; the whole array is never in memory at once
}

API

Formats

Beyond JSON (default), the streaming array pair also speaks XML, binary MessagePack, and binary CBOR — same flat memory, separate helper classes:

FlatXml.EncodeArray(items, stream);
await foreach (var row in ...) { }              // FlatXml.DecodeArray(stream)
FlatMsgPack.EncodeArray(items, stream);          // FlatMsgPack.DecodeArray(stream)
FlatCbor.EncodeArray(items, stream);             // FlatCbor.DecodeArray(stream)

MessagePack and CBOR are byte-identical across all six flatwire languages (see the conformance matrix).

Checked streams

Partial-stream failure semantics: wrap a streamed array in an envelope whose terminal status is written last, so the consumer distinguishes clean completion, an in-band producer error after N rows, and truncation.

using FlatWire;

FlatChecked.EncodeCheckedArray(rows, stream);   // writes ...,"complete":true} last

try
{
    foreach (var row in FlatChecked.DecodeCheckedArray<Row>(stream))
        Handle(row);
}
catch (CheckedStreamException e) { /* producer failed after N rows */ }
catch (TruncatedStreamException) { /* stream ended without a terminal status */ }

The envelope is plain JSON, so a checked stream written in any flatwire language decodes in every other. See docs/FAILURE.md.

HTTP adapter

Stream a large collection to any Stream with flat memory; wire it into an ASP.NET Minimal API via the built-in Results.Stream:

app.MapGet("/rows", () =>
    Results.Stream(
        stream => { FlatHttp.WriteArray(GetRows(), stream, "cbor"); return Task.CompletedTask; },
        FlatHttp.MediaTypes["cbor"]));            // json | xml | msgpack | cbor

See docs/ADAPTERS.md.

License

Apache-2.0 — see the repository.

Benchmarks

See the live benchmark dashboard and the cross-language summary.

Changelog

See CHANGELOG.md.