GrindCore.SharpCompress 0.40.2-alpha

GrindCore.SharpCompress

GrindCore.SharpCompress is an enhanced version of the popular SharpCompress library that integrates GrindCore native compression technology. This project delivers extensive native compression support built using the System.IO.Compression pattern, utilizing compression algorithms directly from their original C implementations.

This enhanced fork replaces GZip, LZMA, Deflate, ZStandard, LZ4, and Brotli implementations with native C streams from GrindCore, providing significant performance improvements while maintaining full API compatibility.

This compression library supports .NET 9, .NET 8, .NET 6, .NET Standard 2.0+, and .NET Framework 4.8+ and can unrar, un7zip, unzip, untar, unbzip2, ungzip, unlzip with forward-only reading and file random access APIs. Write support for zip/tar/bzip2/gzip/lzip are implemented with enhanced native performance.

The library maintains support for non-seekable streams so large files can be processed on the fly (i.e., download streams), now with native-level performance.

⚠️ First Release Notice: This is the initial release of GrindCore.SharpCompress. While tested with comprehensive unit tests, there may be some rough edges as the library gains real-world usage. Please report any issues encountered to help improve the library.

🔧 Native C Integration

GrindCore.SharpCompress provides:

• Native compression streams built using the System.IO.Compression pattern
• Integrated compilation - compression algorithms from original C authors are compiled directly into this project stack
• No external dependencies - pure native C implementations with no third-party DLLs required
• Multiplatform Support: Native libraries for Windows (x64/x86/ARM64), Linux (x64/ARM64/ARM), and macOS (x64/ARM64)
• Easy algorithm updates - C code can be updated with the latest algorithm versions
• Full API compatibility with existing SharpCompress usage

Original SharpCompress Comparison

• Native Performance: All major compression algorithms now use native C implementations via GrindCore
• AOT Compatible: Full support for Ahead-of-Time compilation scenarios
• Enhanced Stream Management: Advanced buffer management with precise stream position correction
• API Compatibility: Drop-in replacement for existing SharpCompress usage
• Framework Support: .NET 9, .NET 8, .NET 6, .NET Standard 2.0+, .NET Framework 4.8+

📊 Native Stream Classes

The following stream classes use GrindCore native implementations:

• DeflateStream: Native Deflate compression with ZLib-NG v2.2.1 (9 levels)
• GZipStream: Enhanced GZip with native performance and metadata support (9 levels)
• ZlibStream: Direct ZLib implementation with advanced buffer correction (9 levels)
• LzmaStream: High-performance LZMA/LZMA2 with Block and Solid modes (9 levels)
• ZStandardStream: Compression with (22 levels)
• LZ4Stream: Fast compression with (12 levels)
• BrotliStream: Web-optimized compression with (11 levels)

🔧 Advanced Features

LZMA Compression

• 7-zip LZMA/2: The algorithms used internally by 7-Zip, not the SDK versions - with modifications to make them .NET Stream compatible
• LZMA2 Block Mode: Configurable block sizes for optimal compression/speed balance
• LZMA2 Solid Mode: Maximum compression efficiency for archive scenarios (CompressionBufferSize = -1)

Buffer Management

• Priority System: WriterOptions.CompressionBufferSize → ReaderOptions.BufferSize → IStreamStack.DefaultBufferSize → Algorithm Default
• Algorithm-Specific Optimization: Automatic buffer sizing based on compression type
• Memory Scaling: Dynamic buffer allocation from 64KB upwards based on data characteristics
• Stream Position Correction: Precise handling of buffer overreads with byte-accurate positioning

📊 Format Support

Legacy Format Support

• Shrink, Implode, Reduce (1-4): Full decompression support
• PPMd, Explode: Advanced legacy algorithm handling
• Multi-volume archives: RAR and ZIP support

🌟 GrindCore: Standalone Library

GrindCore is also available as a standalone library, providing:

• Native compression streams built using the System.IO.Compression pattern
• Framework Support: .NET Framework 3.5 through .NET 9.0
• Platform Support: All .NET platforms with native library auto-selection
• Original Authors' C Code: Direct integration of compression algorithms from their creators
• No Dependencies: Pure native C implementations without third-party dependencies
• Algorithm Updates: C code can be updated with the latest algorithm versions

The standalone GrindCore NuGet package provides comprehensive support across all .NET frameworks and platforms.

Project dependency structure: GrindCore.SharpCompress.dll → GrindCore.net.dll (.NET streams wrapper) → GrindCore.dll (Native C library)

Need Help?

Post Issues on Github!

Check the Supported Formats and Basic Usage.

🛠️ Installation

<PackageReference Include="SharpCompress.GrindCore" Version="x.x.x" />

Or install via Package Manager Console:

Install-Package SharpCompress.GrindCore

📊 Performance Benefits

GrindCore provides measurable performance improvements:

• Compression Speed: Significantly faster compression for supported algorithms
• Decompression Speed: Improved decompression performance
• Memory Efficiency: Optimized buffer management reduces memory overhead
• CPU Utilization: Native implementations leverage modern CPU instruction sets (AVX2, SSE4)

Proven Performance Metrics

Based on comprehensive testing across multiple scenarios:

• LZ4: 400+ MB/s compression, 1500+ MB/s decompression
• ZStandard Level 6: 100+ MB/s compression, 400+ MB/s decompression
• LZMA2 Solid: 95%+ compression ratio on text data
• Deflate/GZip: 3-5x performance improvement over pure C#
• Brotli Level 9: Excellent web compression with 85%+ ratio on text

🔧 Migration from Original SharpCompress

This fork maintains 100% API compatibility. Simply replace your SharpCompress package reference: <PackageReference Include="GrindCore.SharpCompress" Version="x.x.x" /> No code changes required! Your existing SharpCompress code will automatically benefit from GrindCore's 10x performance improvements.

🏗️ Technical Architecture

GrindCore Integration

The enhanced implementation introduces stream classes that wrap GrindCore's native compression:

• DeflateStream: Native Deflate compression via GrindCore (9 levels)
• GZipStream: Enhanced GZip with native performance and metadata support (9 levels)
• ZlibStream: Native ZLib implementation with advanced buffer management (9 levels)
• LzmaStream: High-performance LZMA/LZMA2 with Block and Solid modes (9 levels)
• ZStandardStream: Superior compression ratios (22 levels)
• LZ4Stream: Ultra-fast compression (12 levels)
• BrotliStream: Web-optimized compression (11 levels)

These implementations provide:

• Dual-mode support: Both read-to-compress and write-to-compress patterns
• Stream position correction: Precise handling of buffer overreads
• Memory optimization: Efficient buffer reuse and management
• Error handling: Robust exception handling with detailed diagnostics

Advanced Usage Examples

// Enhanced buffer control with priority system

var options = new WriterOptions(CompressionType.ZStandard, level: 6)
{
    CompressionBufferSize = 2 * 1024 * 1024 // 2MB buffer for optimal performance
};

// LZMA2 Solid Mode for maximum compression
var solidOptions = new WriterOptions(CompressionType.LZMA2, level: 9)
{
    CompressionBufferSize = -1 // Solid mode
};

// Stream-specific optimization
using var stream = new ZStandardStream(output, CompressionMode.Compress, level: 12)
{
    DefaultBufferSize = 256 * 1024 // 256KB working buffer
};

Platform Support

Native libraries are automatically selected for:

• Windows: x64, x86, ARM64
• Linux: x64, ARM64, ARM
• macOS: x64, ARM64

🌐 Framework Compatibility

• .NET: 9.0, 8.0, 6.0
• .NET Standard: 2.0+
• .NET Framework: 4.8.1, 4.8
• NativeAOT: Complete compatibility with native compilation

💡 Note: For broader framework support (.NET Framework 3.5+), use the standalone GrindCore library directly.

🚧 Development Status & Future

GrindCore.SharpCompress represents one year of intensive development to create the first native compression library built the System.IO.Compression way. This groundbreaking achievement required:

• Deep integration of multiple compression algorithm implementations
• Extensive cross-platform native library development
• Comprehensive testing across all supported .NET frameworks
• Advanced buffer management and stream positioning systems
• API compatibility maintenance with the original SharpCompress library

Author & Contributions

The author has been a key contributor to the original SharpCompress project and continues to support its development. Any future SharpCompress enhancements that are compatible with the managed original repository will be submitted as pull requests to prevent the projects from diverging unnecessarily and to ensure the broader community benefits from improvements.

Work is ongoing to further enhance performance, add new compression algorithms, and refine the integration. Your feedback and contributions help drive continued improvements.

A Simple Request

This GrindCore edition builds upon the excellent foundation of the original SharpCompress library while delivering unprecedented native performance. As this is a first release after a year of development, please provide feedback on:

• Performance improvements observed in your applications
• Any issues encountered with the native implementations
• Platform-specific behavior across different operating systems
• Integration experiences with existing codebases

Your input is invaluable for refining and improving this enhanced version.

For performance-related issues or questions specific to GrindCore integration, please mention this in your issue reports.

Please do not email directly for help. Use GitHub issues for all support requests.

Want to contribute?

Contributions are always welcome! Areas of particular interest for this fork include:

• Performance benchmarking and optimization
• Platform-specific testing and validation
• Additional GrindCore algorithm integration
• Documentation improvements
• Real-world usage feedback and testing

TODOs (Enhanced Edition)

• More native algorithms: BZip2, PPMd, Rar decoding etc
• Dictionary support
• Various bugfixes and enhancements to SharpCompress (and therefore GrindCore.SharpCompress)

Version Log

GrindCore.SharpCompress - Current (First Release)

• New: Revolutionary first-of-its-kind native compression integration
• New: 10x Native Performance via GrindCore for all supported algorithms
• New: Native GZip/Deflate performance via GrindCore ZLib-NG v2.2.1 (levels 1-9)
• New: Native LZMA/LZMA2 performance via GrindCore v25.1.0 (levels 1-9)
• New: Native ZStandard compression via GrindCore v1.5.7 (levels 1-22)
• New: Native LZ4 compression via GrindCore v1.10.0 (levels 1-12)
• New: Native Brotli compression via GrindCore v1.1.0 (levels 1-11)
• New: LZMA2 Block and Solid modes with configurable block sizes (levels 1-9)
• New: Multiplatform native library support (Windows/Linux/macOS on x64/ARM64/ARM/x86)
• New: AOT compilation compatibility
• Enhanced: Stream position management with precise buffer correction
• Enhanced: Memory efficiency improvements across all native implementations
• Enhanced: Intelligent buffer management with priority system
• Maintained: Full API compatibility with original SharpCompress
• Achieved: One year of development culminating in this groundbreaking release

📜 License

This enhanced version maintains the same licensing as the original SharpCompress project.

GrindCore Integration: Licensed under MIT License
Native Libraries: Various licenses as specified in GrindCore documentation

🔗 Related Projects

• GrindCore Native: The underlying multiplatform native C library
• GrindCore.net: The dotnet native wrapper providing streams for .NET Framework 3.5 through .NET 9.0
• GrindCore NuGet: Nuget package for the GrindCore dotnet wrapper
• Original SharpCompress: The foundational library this fork enhances