TokenVector.Numerics 1.1.1

TokenVector.Numerics

🇬🇧 English | 🇻🇳 Tiếng Việt

<p align="center"> <img src="assets/icon.png" alt="TokenVector" width="128" /> </p>

<h1 align="center">TokenVector.Numerics</h1>

<p align="center"> <strong>Enterprise Tensor & Mathematical Super-Library</strong><br/> Numerics runtime for the <a href="https://github.com/nguyenhungtran18/TokenVector">TokenVector</a> language </p>

TokenVector.Numerics.dll is a high-performance, standalone mathematical library for multidimensional tensors, Automatic Differentiation (Autograd Engine), analytical linear algebra, optimization, spline interpolation, computational physics, quantitative finance, astrodynamics, quantum computing, post-quantum cryptography, and non-Euclidean geometry built for the TokenVector ecosystem and compiler (a statically-typed native language compiling directly to .NET CIL AOT).

The library serves as the Grand Unified Runtime Math & Tensor Engine, covering multidimensional tensors, automatic differentiation (Autograd), linear algebra, matrix functions, nonlinear optimization, splines, statistical distributions & hypothesis tests, DSP & FFT signal processing, astrodynamics, quantitative finance, computational physics, quantum state simulation, lattice cryptography, structural biology, spectral graph neural networks, and hyperbolic geometry.

📋 Key Features of TokenVector.Numerics

🧪 Verification & Quality Assurance

Reproducible today — exact-arithmetic mathlib/

The mathlib/ modules are pure TokenVector with no tv import, so they compile and run end-to-end with the native compiler. Re-verified September 25, 2026:

tkvc build mathlib/bf_bigfloat.tkv      --out bf_bigfloat.exe      && ./bf_bigfloat.exe
#    t1_sqrt2(100 cs)  t2_pi_chud(100 cs)  t3_e(100 cs)  t4_arith  t5_div  t6_bigmul
#    t7_bignum_neg  t8_precision_borders        -> PASS 8 / 8 - bigfloat OK

tkvc build mathlib/nt_number_theory.tkv --out nt_number_theory.exe && ./nt_number_theory.exe
#    t1..t7  +  t8_i64_boundaries  t9_large_factorization
#                                            -> PASS 9 / 9 - number_theory OK

Defects found and fixed in this pass

Regression tests t7_bignum_neg, t8_precision_borders, t8_i64_boundaries and t9_large_factorization were added to lock these fixes in.

Scope note. The 36/36 row is a source-level algorithm review against independent oracles — not a native execution of linalg/fft/crypto_graph, which the compiler limitation below currently prevents. The 8/8 and 9/9 rows are genuine native runs.

Known limitation — the 175-check stdlib smoke suite

The v1.1.0 release record is 84/84 runtime unit tests, 175/175 .tkv smoke checks and 354/354 numeric-surface coverage (TEST_REPORT.md). The smoke command quoted there

tkvc build tests/tokenvector/smoke_tests.tkv --entry main --out smoke.exe

does not reproduce on tkvc builds that ship without a runtime-path option: the compiler resolves import tv against its own bundled directory and aborts with File khong co ham top-level nao co annotation kieu DSL, while the -r src/tokenvector form referenced by the file header is no longer accepted by tkvc build. Treat 175/175 as the v1.1.0 record, and use the two mathlib suites above as the reproducible check.

See src/tokenvector/README.md for the module map, coverage tables, and benchmarks.

⚡ Performance & Benchmarks

Benchmarks below are measured on the real TokenVector compiler (tkvc.exe, CIL/native): pure-TokenVector kernels — same algorithms as this stdlib (cache-tiled 32×32 matmul, one-sided Jacobi SVD, radix-2/Bluestein FFT) — compiled to standalone .exe files and timed against NumPy 2.5.2 on the same Windows x86_64 machine, same repetition counts per side, best of 3, process startup (~26 ms, measured with a no-op exe) subtracted:

Numeric parity is verified per kernel against NumPy: matmul/add checksums (Σ, Σx²) agree to float64 limits, SVD singular values differ from LAPACK by 2.4e-14, FFT sample coefficients (X[1], X[N/2]) match to ~1e-12.

🔑 How to read the ratios:

• NumPy is a hand-tuned C library (SIMD/AVX2, LAPACK, pocketfft). The compiled TokenVector gap comes from tkvc's boxed-integer arithmetic (TkvInt struct with a BigInteger fast path) and List<T> element storage — not from interpretation.
• These are compiled figures: the earlier interpreter-on-interpreter harness numbers (400–3000×) were 3–250× slower than this on the same kernels.
• The measurement is reproducible: bench_test.tkv (TokenVector source) is compiled with tkvc.exe build bench_test.tkv --out bench_test.exe, split into one kernel per exe, and timed by a small external wrapper.

🚀 Quick Start & Usage

import tv
from tv.core import from_array
import tv.autograd as ag

# 1. Zero-copy multidimensional tensor slicing & broadcasting
a = from_array([1, 2, 3, 4, 5, 6], [2, 3])
b = from_array([10, 20, 30], [1, 3])
c = tv.ops.add(a, b)                     # Stride-0 broadcasting -> [2, 3]
view = a.slice([[0, 2, 1], [1, 3, 1]])   # zero-copy view ~ a[0:2, 1:3]

# 2. Dynamic compute graph autograd
x = ag.Tensor.full(3.0, [1])
x.requires_grad = True
y = (x * x) + (2.0 * x) + 1.0
y.backward()
print(x.grad.get([0]))                   # dy/dx = 2*3 + 2 = 8.0

# 3. Train a multi-layer perceptron (MLP)
model = ag.Sequential([ag.Linear(2, 8), ag.Linear(8, 1)])
optimizer = ag.AdamW(model.parameters(), lr=0.05)

🛠️ Compiling with TokenVector Language (tkvc.exe)

To compile native TokenVector (.tkv / .tv) programs using TokenVector.Numerics.dll:

1. Clone the official TokenVector repository to obtain tkvc.exe and standard libraries (stdlib):

   git clone https://github.com/nguyenhungtran18/TokenVector.git
   

2. Compile your TokenVector program into a standalone native executable:

   ./tkvc.exe main.tkv -r TokenVector.Numerics.dll -o app.exe
   

3. Execute the compiled binary:

   ./app.exe
   

For comprehensive documentation across all 34 mathematical categories and language grammar, check:

• 📖 User Guide (English) | Hướng dẫn sử dụng (Tiếng Việt)
• 📐 TokenVector Syntax Specification | Đặc tả cú pháp TokenVector
• 🏛️ TokenVector Official Compiler Repository