RecursiveAI.OneCompress 0.2.0-preview.1

onearchive-csharp

Compression codecs and archive formats for .NET, written from first principles in managed code. No native dependencies; the compressed bytes are identical on every platform.

Two packages:

• RecursiveAI.OneCompress - codecs and checksums.
• RecursiveAI.OneArchive - archive containers and a simple Archive API, built on OneCompress.

Status: 0.2.0-preview.1. OneCompress has Zstandard (frames, skippable frames, dictionaries and dictionary training, long-distance matching, multithreaded compression in fixed-size jobs, the seekable format), LZ4 (block and frame), Deflate, zlib and gzip, LZMA, LZMA2, xz (with the branch and delta filters) and lzip, the bzip2 decoder, and CRC32, CRC32C, CRC64, Adler-32 and xxHash64. OneArchive reads and writes tar and zip, reads 7z, and handles single compressed streams. Everything is preview: see below.

byte[] packed = Zstd.Compress(data, new ZstdCompressionOptions { Level = 19 });
byte[] again = Zstd.Decompress(packed);

Build

Requires the .NET 10 SDK (see global.json).

build/ci.sh              # build, test (Checked configuration), NativeAOT/trim smoke
build/ci.sh wasm         # browser-wasm build and a headless run under Node (needs the wasm-tools workload)

Build output goes to artifacts/ (ignored by git). Set ONEARCHIVE_ARTIFACTS to put it elsewhere, for example one directory per git worktree.

Preview codecs

Zstandard, LZ4, Deflate, zlib and gzip are preview while their speed is below the reference tools'. Their compression ratio and byte-identical output are already at the target; speed is what is still being tuned. The bzip2 decoder meets its target and stays preview with the rest of this release. The LZMA family is preview until it has been measured on the benchmark machines. The archive layer is preview while the containers are being added and the formats' interop and fuzz coverage grows. PPMd, BCJ2, Deflate64 and encryption are not supported: archive entries that use them are listed, and opening them is a typed "unsupported method" error. Their public types carry [Experimental], so using them is a compiler error until you opt in per codec:

<PropertyGroup>
  <NoWarn>$(NoWarn);ONECOMPRESS001;ONEARCHIVE001</NoWarn>
</PropertyGroup>

or #pragma warning disable ONECOMPRESS001 around the code that uses it. The marker comes off once the codec meets its speed target.

Where they stand

The target for every codec: compressed size within 1% of the reference tool, and compress and decompress speed at least 0.87 of the reference tool's, at every level. Measured memory to memory against zstd 1.5.7, lz4 1.10.0, libdeflate 1.26 (one call), zlib-ng 2.3.3 (streaming) and bzip2 1.0.8, single-threaded unless noted, on Silesia and enwik8 and on small game snapshot payloads with trained dictionaries. Speed is ours divided by the reference (1.00 = same speed). x64 is an AMD Ryzen 9 7950X3D on Linux; arm64 is an Apple M2 on macOS. .NET 10.0.402 with the JIT.

Output is byte-identical across x64 and arm64, the JIT and NativeAOT, with and without hardware intrinsics, at any thread count, and in the browser (WebAssembly).

Archives

Archive.Extract("data.tar.zst", "out");                         // safe by default: no writes outside "out", limits on
Archive.Create("backup.tar.zst").Add("assets/").Save();          // format and compression from the extension
using var archive = Archive.Open("data.tar");                  // entries, random access when the source can seek

tar (ustar, pax, GNU, sparse files) and zip (store, Deflate and Zstandard; zip64, data descriptors; LZMA, xz and bzip2 entries read) are read and written; 7z is read (Copy, Deflate, BZip2, LZMA, LZMA2, Delta, the branch filters, Zstandard; solid folders, encoded headers). Single streams (.gz, .zst, .lz4, and .bz2 for reading) and tar inside them (.tar.gz, .tar.zst, ...) are one archive. The other containers plug into the same reader and writer interfaces. The model, the safety defaults and limits, and the tar coverage are in docs/archive.md.

Layout

• src/OneCompress, src/OneArchive - the two packages.
• tests/ - xUnit tests.
• fuzz/ - in-process mutational fuzz driver and targets.
• bench/ - benchmarks; the only project that may reference comparison libraries.
• smoke/aot-trim, smoke/wasm - NativeAOT/trim and browser-wasm smoke apps.
• build/ci.sh - the script CI and developers run.

Licence

Licensed under either of MIT or Apache-2.0, at your option. Copyright (c) Recursive AI, LLC.