HeatshrinkDotNet 1.0.0

HeatshrinkDotNet

A C# port of heatshrink, the LZSS-based compression library designed for embedded and real-time systems with very small memory footprints. The port keeps the original's streaming sink/poll API and produces byte-identical output, so data is interchangeable with the C implementation (given matching window/lookahead parameters).

The library targets netstandard2.0 and net10.0. Sink and Poll accept ReadOnlySpan<byte> / Span<byte> as well as arrays, so slices, stackalloc'd, and pooled buffers can be used without copying (the netstandard2.0 target uses the System.Memory package for span support).

Configuration

Both the encoder and decoder are configured with two parameters that must match between compression and decompression:

• windowBits (4–15): the sliding window is 2^windowBits bytes. Larger windows find more repetition and compress better, at the cost of memory.
• lookaheadBits (3 to windowBits - 1): the maximum back-reference length is 2^lookaheadBits bytes.

The decoder additionally takes an input buffer size (any positive value; it only affects how much compressed data can be sunk at once).

Usage

The API is a streaming state machine: Sink feeds input in, Poll drains output, and Finish signals the end of the stream. Buffers can be as small as one byte.

using HeatshrinkDotNet;

static byte[] Compress(byte[] input)
{
    var encoder = new HeatshrinkEncoder(windowBits: 10, lookaheadBits: 5);
    var output = new System.IO.MemoryStream();
    var chunk = new byte[256];

    var sunk = 0;
    var finished = false;
    while (!finished)
    {
        if (sunk < input.Length)
        {
            encoder.Sink(input, sunk, input.Length - sunk, out var count);
            sunk += count;
        }

        EncoderPollResult pres;
        do
        {
            pres = encoder.Poll(chunk, out var polled);
            output.Write(chunk, 0, polled);
        } while (pres == EncoderPollResult.More);

        if (sunk == input.Length)
            finished = encoder.Finish() == EncoderFinishResult.Done;
    }
    return output.ToArray();
}

static byte[] Decompress(byte[] compressed)
{
    var decoder = new HeatshrinkDecoder(inputBufferSize: 256, windowBits: 10, lookaheadBits: 5);
    var output = new System.IO.MemoryStream();
    var chunk = new byte[256];

    var sunk = 0;
    var finished = false;
    while (!finished)
    {
        if (sunk < compressed.Length)
        {
            decoder.Sink(compressed, sunk, compressed.Length - sunk, out var count);
            sunk += count;
        }

        DecoderPollResult pres;
        do
        {
            pres = decoder.Poll(chunk, out var polled);
            output.Write(chunk, 0, polled);
        } while (pres == DecoderPollResult.More);

        if (sunk == compressed.Length)
            finished = decoder.Finish() == DecoderFinishResult.Done;
    }
    return output.ToArray();
}

Reset() returns an encoder or decoder to its initial state so the instance can be reused. Instances are not thread-safe; use one per stream.

Both constructors accept an optional Microsoft.Extensions.Logging.ILogger as the last parameter. Diagnostic output (mirroring the C library's LOG statements) is emitted at Trace level, with allocation messages at Debug; when no logger is passed, or the level is disabled, logging costs a single guard check per site.

Tests

The test suite is a port of upstream heatshrink's, using xUnit v3 on .NET 10:

dotnet test

This includes two deterministic pseudo-random round-trip sweeps (FuzzingSingleByteSizes / FuzzingMultiByteSizes) covering thousands of size/seed/buffer combinations, and a CsCheck port of upstream's theft-based property tests (round-trip integrity, the 9/8 worst-case size bound, and encoder/decoder liveness under arbitrary input).

A separate project, HeatshrinkDotNet.InteropTest, compiles the vendored upstream C sources into the reference heatshrink CLI and verifies the two implementations produce byte-identical, mutually decodable streams — both on fixed cases and via a CsCheck differential-fuzzing property with the C implementation as the oracle. The C build needs a POSIX cc; on Windows these tests skip themselves.

One known, tested divergence: at windowBits 15 the C encoder never finds a match at all — both its search paths (indexed and brute-force) use int16_t position arithmetic, and the input region starts at buffer offset 32768, so every position reads as negative and the search loops exit immediately. The C implementation therefore emits pure literal output (a 9/8 expansion) at its own maximum window setting, while this port compresses normally. The streams remain valid and decode interchangeably in both directions; outputs are byte-identical for windowBits up to 14, which works exactly because 32768 buffer positions max out int16_t at 32767. Known upstream as atomicobject/heatshrink#86 (partial fix proposed in #88, which does not cover the brute-force path). The same 16-bit arithmetic also makes the C encoder hang once 32K of input is buffered at windowBits 15 (#55); this port does not share that, and the C decoder — which is unaffected — decodes this port's windowBits 15 streams byte-exactly (covered by the interop tests).

Fuzzing

HeatshrinkDotNet.Fuzz provides coverage-guided fuzzing (SharpFuzz + libFuzzer) with two harnesses: decoder feeds arbitrary bytes to the decoder, which must never throw, hang, or over-produce; roundtrip checks that any payload compresses within the 9/8 bound and decompresses back to itself. The first input bytes select the window/lookahead configuration, so the fuzzer explores those too.

HeatshrinkDotNet.Fuzz/fuzz.sh decoder 300

The pipeline runs natively on Linux; on macOS the script runs it in a Docker container (libFuzzer's coverage transport is ELF-only). The corpus persists in HeatshrinkDotNet.Fuzz/corpus/, and crashing inputs can be replayed with dotnet run -- replay <harness> <file>.

License

This port is licensed under the LGPL-3.0 (see LICENSE).

The original heatshrink C library is copyright (c) 2013-2015 Scott Vokes and licensed under the ISC license; its notice is preserved in LICENSE.heatshrink.

The original C# port was written by Shindo in 2019.