StreamHash 1.2.0

StreamHash

StreamHash is a high-performance, memory-efficient streaming hash library for .NET 10+. It provides incremental/streaming implementations of popular non-cryptographic hash algorithms that traditionally require loading entire files into memory.

🎯 Why StreamHash?

Many popular hash algorithms (MurmurHash, CityHash, SpookyHash, etc.) don't have official streaming APIs. This means hashing a 10GB file requires 10GB of RAM! StreamHash solves this by providing streaming implementations that process data in chunks, using only ~1MB of memory regardless of file size.

✨ Features

• 🚀 Memory Efficient: Hash multi-gigabyte files with minimal memory footprint
• ⚡ High Performance: SIMD-optimized implementations (SSE2, AVX2, AVX-512)
• 🔄 Streaming API: Process data incrementally with Update() and Finalize()
• 📦 Zero Allocations: Hot paths are allocation-free using Span<T>
• 🧪 Thoroughly Tested: Validated against official test vectors
• 📖 Fully Documented: XML docs, examples, and algorithm references

📊 Supported Algorithms

16 algorithms total with 532 tests!

* xxHash wrappers use System.IO.Hashing for IStreamingHash compatibility

🚀 Quick Start

Installation

dotnet add package StreamHash --version 1.2.0

Basic Usage

using StreamHash.Core;

// Hash a file incrementally
using var hasher = new MurmurHash3_128();
using var stream = File.OpenRead("large-file.bin");

byte[] buffer = new byte[1024 * 1024]; // 1MB buffer
int bytesRead;

while ((bytesRead = await stream.ReadAsync(buffer)) > 0) {
    hasher.Update(buffer.AsSpan(0, bytesRead));
}

var hash = hasher.Finalize();
Console.WriteLine($"Hash: {Convert.ToHexStringLower(hash)}");

One-Shot API

// For small data that fits in memory
var hash = MurmurHash3.ComputeHash128(data);

📖 Documentation

• 📚 Algorithm Reference
• 🔧 API Documentation
• 📈 Performance Benchmarks
• 🎓 Usage Guides

🏗️ Building

# Clone the repository
git clone https://github.com/TheAnsarya/StreamHash.git
cd StreamHash

# Build
dotnet build StreamHash.sln

# Run tests
dotnet test

# Run benchmarks
dotnet run -c Release --project benchmarks/StreamHash.Benchmarks

📊 Benchmarks

Benchmarks comparing StreamHash to reference implementations:

| Method              | File Size | Memory    | Throughput |
|---------------------|-----------|-----------|------------|
| MurmurHash3 (ref)   | 1 GB      | 1,024 MB  | 3.2 GB/s   |
| MurmurHash3 (stream)| 1 GB      | 1 MB      | 3.1 GB/s   |
| CityHash (ref)      | 1 GB      | 1,024 MB  | 4.5 GB/s   |
| CityHash (stream)   | 1 GB      | 1 MB      | 4.3 GB/s   |

🤝 Contributing

Contributions are welcome! Please read our Contributing Guide for details.

📄 License

This project is released into the public domain under The Unlicense. Do whatever you want with it.

🙏 Acknowledgments

• SMHasher - MurmurHash reference implementation
• CityHash - Google's CityHash
• SpookyHash - Bob Jenkins' SpookyHash
• SipHash - Reference SipHash implementation