TokenVector.Numerics 1.1.0

TokenVector.Numerics

🇬🇧 English | 🇻🇳 Tiếng Việt

<p align="center"> <img src="assets/icon.png" alt="TokenVector" width="128" /> </p>

<h1 align="center">TokenVector.Numerics</h1>

<p align="center"> <strong>Enterprise Tensor & Mathematical Super-Library</strong><br/> Numerics runtime for the <a href="https://github.com/nguyenhungtran18/TokenVector">TokenVector</a> language </p>

TokenVector.Numerics.dll is a high-performance, standalone mathematical library for multidimensional tensors, Automatic Differentiation (Autograd Engine), analytical linear algebra, optimization, spline interpolation, computational physics, quantitative finance, astrodynamics, quantum computing, post-quantum cryptography, and non-Euclidean geometry built for the TokenVector ecosystem and compiler (a statically-typed native language compiling directly to .NET CIL AOT).

The library serves as the Grand Unified Runtime Math & Tensor Engine, covering multidimensional tensors, automatic differentiation (Autograd), linear algebra, matrix functions, nonlinear optimization, splines, statistical distributions & hypothesis tests, DSP & FFT signal processing, astrodynamics, quantitative finance, computational physics, quantum state simulation, lattice cryptography, structural biology, spectral graph neural networks, and hyperbolic geometry.

📋 Key Features of TokenVector.Numerics

🧪 Verification & Quality Assurance

All 84/84 automated unit tests passed with zero failures in Release mode:

dotnet test TokenVector.Numerics.sln -c Release

Passed!  - Failed: 0, Passed: 84, Skipped: 0, Total: 84, Duration: 179 ms - TokenVector.Numerics.Tests.dll (net8.0)

New in v1.1.0 — TokenVector stdlib (.tkv): the library is now also published in the TokenVector language with 100% NumPy library-surface coverage (354/354) and its own verification toolchain:

python tests/tokenvector/tkv_harness.py
#    TokenVector stdlib smoke tests: passed=175, failed=0
python tests/tokenvector/numpy_coverage_audit.py
#    matched in .tkv stdlib : 354 (100%) | truly missing: 0

See src/tokenvector/README.md for the module map, coverage tables, and NumPy benchmark.

⚡ Performance & Benchmarks

Benchmarked on AMD Ryzen / Intel x86_64 (Release build, .NET 8 LTS, Native SIMD AVX2/FMA, 100% Core scaling via Parallel.For):

🔑 Key Acceleration Pillars:

• Hardware SIMD (AVX2 & FMA): Processes 8 FP32 values per CPU cycle in hardware vector registers with fused multiply-add.
• $32 \times 32$ Cache-Tiling: Keeps matrix sub-blocks within ultra-fast L1 Data Cache (1–4 ns latency), eliminating memory wall cache misses.
• True No-GIL Multithreading: Scales compute tasks linearly across 100% of physical CPU cores via Parallel.For.
• Zero-GC & Direct Pointers: Leverages unmanaged TensorBuffer<T>, Span<T>, and in-place buffer reuse without Garbage Collector pauses.
• Zero-RAM Memory-Mapping: Uses OS kernel mmap to slice multi-gigabyte tensors from NVMe disk with $< 1\text{ ms}$ latency and 0 MB RAM overhead.

📐 Benchmark Methodology & Computational Basis:

• MatMul ($1024 \times 1024$, $2.15\text{ GFLOPs}$): Naive 3-loop scalar causes severe L1/L2 cache misses ($\sim 14.5\text{ GFLOPS} \rightarrow 148.2\text{ ms}$). TokenVector.Numerics utilizes $32 \times 32$ cache tiling to lock data in 4KB L1 cache ($>1.5\text{ TB/s}$ bandwidth) combined with AVX2 FMA (16 FLOPs/cycle) and 16-thread Parallel.For ($\sim 275\text{ GFLOPS} \rightarrow \mathbf{7.8\text{ ms}}$).
• 2D FFT ($1024 \times 1024$, $\sim 105\text{ MFLOPs}$): Replaces $O(N^2)$ discrete transforms with Radix-2 Cooley-Tukey + SIMD twiddle factors + multithreaded row/column concurrency ($82.5\text{ ms} \rightarrow \mathbf{6.1\text{ ms}}$).
• Autograd MLP Backward (1000 iter, $B=64, D=128$): Traditional frameworks suffer continuous GC pauses from per-step node allocations ($312.0\text{ ms}$). TokenVector.Numerics utilizes a Zero-Allocation static DAG with in-place unmanaged gradient reuse ($\mathbf{18.4\text{ ms}}$).
• Cosine Similarity ($1\text{M} \times 128\text{-dim}$, $512\text{ MB}$): Single-pass triple-vector AVX2 registers simultaneously compute dot product, normA, and normB, saturating full memory bus bandwidth ($\sim 45\text{ GB/s} \rightarrow \mathbf{11.2\text{ ms}}$).
• Out-of-Core Disk I/O ($10\text{ GB}$ .npy): Eliminates $4.2\text{ s}$ full RAM disk ingestion by using kernel virtual memory page tables to demand-load only accessed 4KB pages in $\mathbf{0.8\text{ ms}}$ with 0 MB RAM footprint.

🚀 Quick Start & Usage

using TokenVector.Numerics.Core;
using TokenVector.Numerics.Autograd;
using TokenVector.Numerics.Autograd.NN;
using TokenVector.Numerics.Autograd.Optim;

// 1. Zero-copy Multidimensional Tensor Slicing & Broadcasting
var a = NDArray<double>.FromArray([1, 2, 3, 4, 5, 6], 2, 3);
var b = NDArray<double>.FromArray([10, 20, 30], 1, 3);
var c = a + b; // Stride-0 Broadcasting -> [2, 3]

// 2. Dynamic Compute Graph Autograd
var x = new Tensor<double>(3.0, requiresGrad: true);
var y = (x * x) + (2.0 * x) + 1.0;
y.Backward();
Console.WriteLine(x.Grad!.Buffer[0]); // dy/dx = 2*3 + 2 = 8.0

// 3. Train a Multi-Layer Perceptron (MLP)
var model = new Sequential<double>(
    new Linear<double>(inFeatures: 2, outFeatures: 8),
    new Linear<double>(inFeatures: 8, outFeatures: 1)
);
var optimizer = new AdamW<double>(model.Parameters(), lr: 0.05);

🛠️ Compiling with TokenVector Language (tkvc.exe)

To compile native TokenVector (.tkv / .tv) programs using TokenVector.Numerics.dll:

1. Clone the official TokenVector repository to obtain tkvc.exe and standard libraries (stdlib):

   git clone https://github.com/nguyenhungtran18/TokenVector.git
   

2. Compile your TokenVector program into a standalone native executable:

   ./tkvc.exe main.tkv -r TokenVector.Numerics.dll -o app.exe
   

3. Execute the compiled binary:

   ./app.exe
   

For comprehensive documentation across all 34 mathematical categories and language grammar, check:

• 📖 User Guide (English) | Hướng dẫn sử dụng (Tiếng Việt)
• 📐 TokenVector Syntax Specification | Đặc tả cú pháp TokenVector
• 🏛️ TokenVector Official Compiler Repository