StreamLZ 1.0.2

StreamLZ

High-performance LZ compression library for .NET with streaming support.

Features

• 6.0 GB/s decompress at level 1, 27% ratio at level 11 (enwik8)
• Single level scale (1-11) — no codec selection needed
• Streaming — SLZ1 frame format supports files of any size
• Sliding window — cross-block match references for better ratio
• Parallel compression — automatic multi-threading with configurable thread limits
• Async — CompressFileAsync, DecompressFileAsync, IAsyncDisposable on SlzStream
• Validation — TryDecompress (non-throwing), IsValidFrame, content checksums
• Zero allocations on the hot path (pooled scratch buffers)
• Native AOT and trimming compatible
• Targets net8.0 and net10.0

Installation

dotnet add package StreamLZ

Quick Start

using StreamLZ;

// Simplest: compress and decompress byte arrays (SLZ1 framed, self-describing)
byte[] compressed = Slz.CompressFramed(data);
byte[] restored = Slz.DecompressFramed(compressed); // no size tracking needed

// Compress / decompress files
Slz.CompressFile("input.txt", "output.slz");
Slz.DecompressFile("output.slz", "restored.txt");

// Stream-based (any size)
Slz.CompressStream(input, output, level: 6);

// Named compression levels
byte[] fast = Slz.CompressFramed(data, SlzCompressionLevel.Fast);
byte[] max = Slz.CompressFramed(data, SlzCompressionLevel.Maximum);

Compression Levels

API

StreamLZ offers three API tiers. Choose based on your use case:

Framed in-memory (simplest — self-describing round-trip)

Uses the SLZ1 frame format. Output includes size metadata so decompression needs no external information. Best for storing/transmitting compressed blobs.

byte[] compressed = Slz.CompressFramed(data);
byte[] restored = Slz.DecompressFramed(compressed);

// Named levels for readability
byte[] fast = Slz.CompressFramed(data, SlzCompressionLevel.Fast);

Raw in-memory (zero-copy — caller manages buffers)

No framing. Caller must track the original size and provide output buffers (including Slz.SafeSpace extra bytes for decompression). Best for hot paths where you control the buffer lifecycle.

int bound = Slz.GetCompressBound(data.Length);
byte[] dst = new byte[bound];
int compSize = Slz.Compress(data, dst, level: 3);

byte[] output = new byte[originalSize + Slz.SafeSpace];
Slz.Decompress(compressed, output, originalSize);

// Non-throwing variant for untrusted data
if (Slz.TryDecompress(compressed, output, originalSize, out int written))
    // success

Important: Raw and framed formats are not interchangeable. Data compressed with Compress must be decompressed with Decompress (not DecompressFramed), and vice versa.

File and stream (any size, SLZ1 framed)

Uses the SLZ1 frame format with a sliding window for cross-block match references. Supports files of any size with bounded memory usage.

// Sync
Slz.CompressFile("input.txt", "output.slz");
Slz.DecompressFile("output.slz", "restored.txt");
Slz.CompressStream(input, output, level: 6);
Slz.DecompressStream(input, output);

// Async
await Slz.CompressFileAsync("input.txt", "output.slz", cancellationToken: ct);
await Slz.DecompressFileAsync("output.slz", "restored.txt", cancellationToken: ct);

// With content checksum for integrity verification
Slz.CompressFile("input.txt", "output.slz", useContentChecksum: true);

// Limit compression threads (for server workloads)
Slz.CompressFile("input.txt", "output.slz", maxThreads: 4);

SlzStream (GZipStream-style wrapper)

// Compress (supports await using for async disposal)
await using var compressStream = new SlzStream(outputStream, CompressionMode.Compress);
inputStream.CopyTo(compressStream);

// Decompress
await using var decompressStream = new SlzStream(inputStream, CompressionMode.Decompress);
decompressStream.CopyTo(outputStream);

// With options
var options = new SlzStreamOptions
{
    Level = 9,
    UseContentChecksum = true,
    LeaveOpen = true
};
await using var stream = new SlzStream(inner, CompressionMode.Compress, options);

Note: Disposing an SlzStream in compress mode without writing any data produces no output. To get a valid empty SLZ1 stream, write at least one byte, or use CompressFramed(ReadOnlySpan<byte>.Empty).

Validation

bool valid = Slz.IsValidFrame(compressedData);
bool valid = Slz.IsValidFrame(stream); // rewinds if seekable

JIT warmup (optional)

// Called automatically on first use of Slz. Call explicitly at app
// startup to move the ~15ms JIT cost to a predictable point.
Slz.WarmUp();

Comparison vs LZ4, Snappy, Zstd

enwik8 (100 MB text, 3-run median)

silesia (212 MB mixed, 3-run median)

All benchmarks on Intel Arrow Lake-S (Ultra 9 285K), .NET 10, multi-threaded.

License

MIT