ZeroData.Sql 1.5.0

ZeroData

ZeroData is a comprehensive, blazing-fast data platform for .NET, combining in-memory streaming analytics and high-performance RDBMS data access for the Zero Universe ecosystem.

Subsystems:

1. ZeroData.Core: In-memory zero-allocation columnar DataFrame and streaming analytics engine (Apache Arrow IPC / Polars equivalent) with zero external dependencies.
2. ZeroData.Sql: High-performance hybrid RDBMS data access and lightweight ORM, powered by a sovereign native ADO.NET engine with compiled Expression Tree materializers, CPU-register unboxing via ZeroPrimitives.Core, and zero external third-party dependencies. Provides modern ergonomic syntax, fast primary-key lookups, batch deletes/updates, unit of work, and full LINQ to SQL parity.
3. ZeroData.Sql.CodeGen: CLI tool (zerodata-sql-codegen) to reverse-engineer SQL schemas and DBML models into strongly-typed C# entities and SqlContext.

🌟 Key Capabilities

ZeroData.Core (Streaming & Columnar Analytics)

• SIMD Columnar Filter & SelectionMask:
  • Arrow-compliant compressed bitmask filtering (SelectionMask) representing boolean filter results at 1 bit per row.
  • Hardware vector comparison pushdown via ColumnFilterOps (DataFrame.Where(op)), accelerating query scans by up to 15x over naive row-wise filtering.
• Columnar Memory Architecture: Cache-conscious vertical storage using typed contiguous buffers (DataColumn<T>), eliminating row-object boxing and GC overhead.
• Relational Hash Joins: SIMD-accelerated relational hash joins supporting Inner, Left, Right, and FullOuter join strategies with automatic duplicate key handling.
• Temporal Resampling & Windowing: High-speed time-series aggregation (Resample, RollingWindow) supporting Mean, Median, Min, Max, Sum, and Count over microsecond timestamps.
• Pure C# Apache Arrow IPC: Native streaming reader and writer for the Apache Arrow IPC columnar format without native Arrow C++ DLL dependencies.
• Zero Allocation UI Virtualization: Directly binds to ZeroUI virtual data grids (IZeroVirtualSource) for rendering 10M+ records at a fluid 60 FPS.

ZeroData.Sql (Sovereign High-Performance RDBMS ORM)

• Zero Third-Party NuGets: Core ORM depends strictly on ADO.NET (System.Data) and ZeroPrimitives.Core with zero runtime dependencies.
• Direct CPU-Register Unboxing: IL Expression Tree materializer bypasses heap boxing on primitives (int, long, decimal, double, bool, DateTime, Guid) for maximum memory locality.
• Ergonomic & Modern Syntax: Direct CRUD (db.Insert(e), db.Update(e), db.Delete(e), db.Save()), batch operations, and server-side set deletes (db.DeleteById<T>(id), table.DeleteWhere(predicate)).
• Fast Primary Key Lookup: Direct compiled metadata cache lookup (db.Get<T>(id) / db.GetAsync<T>(id)), bypassing Expression Tree compilation.
• Read-Only Zero-Allocation Queries: db.Query<T>() skips snapshot tracking allocations by default for maximum memory efficiency.
• Sovereign Native SQL Power: Full native SQL queries and commands using anonymous object parameters (db.QuerySql<T>(sql, new { ... }), db.ExecuteSql(...)).
• Cross-Framework Compatibility: Standard netstandard2.0 target runs seamlessly on both legacy .NET Framework 4.6.2 - 4.8 and modern .NET 8 / 9 / 10.

📦 Installation

Install via the .NET CLI:

dotnet add package ZeroData.Core

🚀 Quick Start

1. Creating a Columnar DataFrame

using ZeroData.Core;

var df = new DataFrame();
df.AddColumn("Timestamp", new DateTime[] { DateTime.UtcNow, DateTime.UtcNow.AddSeconds(1) });
df.AddColumn("Temperature", new double[] { 72.4, 73.1 });
df.AddColumn("Status", new string[] { "OK", "WARN" });

Console.WriteLine($"Rows: {df.RowCount}, Columns: {df.ColumnCount}");

2. High-Performance Relational Hash Join

var left = new DataFrame();
left.AddColumn("Id", new int[] { 1, 2, 3 });
left.AddColumn("Part", new string[] { "Gear", "Shaft", "Bearing" });

var right = new DataFrame();
right.AddColumn("Id", new int[] { 1, 2, 4 });
right.AddColumn("Price", new double[] { 12.5, 45.0, 8.2 });

// Perform Inner Join on 'Id'
var joined = left.Join(right, "Id", JoinType.Inner);

Console.WriteLine($"Joined RowCount: {joined.RowCount}");

3. Time-Series Resampling

// Downsample high-frequency 1000Hz sensor data to 1-second intervals (Mean aggregation)
var resampled = df.Resample("Timestamp", TimeSpan.FromSeconds(1), AggregationType.Mean);

4. SIMD Columnar Filtering & SelectionMask

using ZeroData.Core;

// Perform hardware-vectorized filter pushdown (> 75.0) returning a bitmask
var mask = df.FilterGreaterThan("Temperature", 75.0);

// Slice dataframe zero-copy using Arrow-compliant SelectionMask
var hotRows = df.Where(mask);
Console.WriteLine($"Hot Rows Detected: {hotRows.RowCount}");

📊 Benchmark & Performance

1. ZeroData.Core (In-Memory Columnar Engine)

Tested on Intel Core i7-13700K (1 Million Rows, Release x64):

2. ZeroData.Sql (Sovereign Native ADO.NET Engine)

Actual verified benchmark results on compiled Expression Tree materializer:

📜 Release History

📄 License

MIT License © 2026 Phong Võ. Part of the ZeroPlatform project.