NumSharp.Build 0.80.0

NumSharp.Build

Build-time IL weaver for NumSharp's [NDScoped] deterministic memory reclamation. Installing this package is the whole opt-in — it wires a post-compile step into your project's build, and it is not a dependency and not contagious: it weaves the project it is installed in and nothing downstream. It ships MSBuild targets + a tool only (no lib/, no dependency entries, build/ never buildTransitive/), so a project that merely references your woven library — by project or by package — is never woven and its own [NDScoped] stays inert. dotnet add package installs it with PrivateAssets="all", which keeps it out of your own package's dependency graph; if you write the reference by hand (or under Central Package Management), add PrivateAssets="all" yourself — dotnet pack refuses to ship NumSharp.Build as a dependency of your package otherwise (error NDW018, with the one-line fix; -p:NumSharpBuildAllowAsDependency=true overrides it on purpose).

What it does

A NumSharp composition method allocates a handful of intermediate NDArrays, keeps one as its result, and drops the rest. Dropped arrays normally wait on the finalizer to return their pooled buffers — in a tight loop that lag means cold allocations. Mark the method [NDScoped] and keep the body exactly as you wrote it:

using NumSharp;

[NDScoped]
public static NDArray Normalize(NDArray a)
{
    var mean = np.mean(a);           // transient
    var std  = np.std(a);            // transient
    var centered = a - mean;         // transient
    return centered / std;           // the one you keep
}

At build time the method is rewritten to open an NDScope — the transients are reclaimed the moment the method exits (exception paths included) while the result survives. Your source keeps its 100 % original body; the scope exists only in the compiled IL.

Synchronous iterators work too, under [NDScoped]. An IEnumerable<NDArray> / IEnumerator<NDArray> (yield return) method is woven through its compiler state machine: one scope spans the whole enumeration, yield returned elements survive (the consumer owns them), and hoisted state is reclaimed at the end of iteration or the enumerator's Dispose() (a foreach break included).

Async methods, async iterators and Task/ValueTask returns use [NDScopedAsync]. An async Task<NDArray> / async ValueTask<NDArray> method, an IAsyncEnumerable<NDArray> async iterator, or a non-async method returning Task/ValueTask is marked [NDScopedAsync] (the companion attribute for shapes that suspend across await). The async ones weave through the state machine — one scope spans the logical invocation, suspended across awaits and resumed on whatever thread continues, so temps handed to an in-flight awaited callee stay alive and everything is reclaimed when the invocation completes; a non-async Task/ValueTask return yields a completed task's result immediately and defers reclamation to an incomplete task's completion. Marking a method with the wrong attribute is a build error (NDW009/NDW010), never a silent unwoven ship.

A retained argument is protected with [NDScopedExit]. A scope reclaims every array it tracked except the return value and out params — so an array handed to something that keeps it (a field store, a long-lived collection, a captured closure) would be disposed under the retainer. Mark the retaining parameter and the weaver detaches the argument from the caller's scope at the callee's entry:

public NDArray Weights { get; private set; }
public void Adopt([NDScopedExit] NDArray w) => Weights = w;   // w survives the caller's scope

It works with or without a method-level scope attribute, covers NDArray / NDArray[] / tuples of NDArrays — nested tuples and NDArray[] components detach recursively — (a property setter's value counts), and is a no-op when there is no ambient scope. An unsupported parameter type is a build error (NDW014). A raw public-field store has no parameter to annotate — route it through a setter or call NDScope.Detach by hand.

The caller's arrays stay alive for the whole call: operand roots. A scope roots only the arrays it tracks (those built while it is open); an operand the caller built is never tracked, and optimized code stops reporting a parameter to the GC at its last use — so a method that reads an operand's raw address and then runs a kernel could have the operand collected under the kernel when the caller passed a temporary. Every method that opens a scope ([NDScoped], or a hand-written NDScope.Open()) therefore copies each NDArray-reaching parameter (and this) into a shadow local at entry and passes it to NDScope.Anchor (a no-code GC.KeepAlive) before every return. An unscoped entry point that hands an operand's address to a kernel carries [NDAnchored] for the same roots without a scope:

[NDAnchored]
public static unsafe double Sum(NDArray a)          // a stays reachable until Sum returns
{
    long n = a.size;                                // (a C-contiguous float64 operand)
    double* p = a.Unsafe.Pointer<double>();         // a's last use: without the root, Sum(np.arange(n) * 1.0)
    double s = 0;                                   // could lose its temporary operand here
    for (long i = 0; i < n; i++) s += p[i];
    return s;
}

[NDAnchored] roots only the parameters (no temporaries: that is [NDScoped]'s job), is not inherited, and is refused on a body-less, async, iterator or scoped method (NDW019). NumSharp's own assembly is woven the same way, so its API keeps the operands you pass it alive whether or not your project installs this package; the package matters for the scopes and the unsafe code in your methods.

On a virtual, abstract or interface member the attributes are a contract the inheritors inherit. Every override and implementation — class overrides through any number of levels, implicit and explicit interface implementations, generic instantiations, and overrides in another assembly of a NumSharp member or of a library that used this weaver — is woven exactly as if it carried the attribute itself, so the scoping is stated once, where the API is declared:

public abstract class Layer
{
    [NDScoped] public abstract NDArray Forward(NDArray x);        // the contract — never woven, never an error
    public abstract void Adopt([NDScopedExit] NDArray weights);   // inherited by position
}

public class Dense : Layer
{
    public override NDArray Forward(NDArray x) { var h = x + 1.0; return h.copy(); } // woven
    public override void Adopt(NDArray w) => _w = w;                                  // w is detached
    private NDArray _w;
}

The nearest declaration up the chain wins, and an override's own attribute wins over it — [NDScopedCovered] on an override opts out of the inherited weave. The analyzer resolves the same graph (no NDW012 on an inheriting override; the target gate reads the declaration's attribute), and while this package is absent it reports NDW013 at every member that would have been woven — an own [NDScoped]/[NDScopedAsync]/[NDScopedExit] and an inherited one alike, the latter a shape no metadata text scan can see because the override never spells the attribute.

A companion Roslyn analyzer catches mistakes at compile time — and it ships with NumSharp itself, not with this package. The NumSharp package carries the analyzer (under its analyzers/dotnet/cs/, applied automatically to any PackageReference compile), so it is already active in every project that can use the attributes — before this weaver is ever installed. It reports a wrong or unsupported target as a build ERROR — in the editor, before the weave runs: the wrong attribute (NDW009 = an async/Task method under [NDScoped], NDW010 = a plain sync method or synchronous iterator under [NDScopedAsync], NDW011 = both attributes), a hidden ref/in egress over any NDArray-carrying shape (NDW002), an unsupported carrier return (NDW003), a body-less method (NDW005), a setter-only property (NDW006), or an out parameter whose NDArray-carrying shape the out-escape cannot yield (NDW015). The IL-only checks (an unrecognized state machine, a tail-call, an out-of-date NumSharp, a bad [NDScopedExit] parameter) stay with the weaver post-compile. A SOURCE-mode consumer (ProjectReference to NumSharp.Core + imported targets — no package, so no auto-apply) opts the analyzer in by referencing the analyzer project with OutputItemType="Analyzer", or by setting $(NumSharpBuildAnalyzerDll) at the built analyzer DLL — the parallel to $(NumSharpBuildToolDll).

With the package absent, [NDScoped] (which ships in NumSharp itself) is inert metadata: the method runs unscoped and transients fall back to the finalizer. NumSharp does not depend on or bundle this package — weaving is an explicit opt-in — so NumSharp's own analyzer (and, where no analyzer runs, an MSBuild scan in its targets) reports NDW013 at every attributed member in that state, naming this package as the fix. Adding or removing the package never changes results — only when buffers are reclaimed.

Install

dotnet add package NumSharp.Build

Requires a project that references NumSharp (that is where NDScope and the attribute live) and a .NET 8+ host runtime for the build tool (it rolls forward to any newer major).

Escape hatches

Documentation

Full reference — which return shapes are woven (tuples, NDArray[], result-struct carriers via INDArrayCarrier, out params), the NDScope API, hand-scoping, and the NDW00x build errors: https://scisharp.github.io/NumSharp/docs/numsharp-build-compiler.html

Part of the SciSharp STACK.

What's new in 0.80.0

We are at 95.2% NumPy API coverage, and everything except datetime, complex64 and other exotic dtypes matches NumPy byte for byte. This release brings NumPy's text dtypes with numpy.strings and numpy.char, numpy.ma, numpy.polynomial, np.emath and 48 new np.* functions. Integration rather than competition goes on, this time inside .NET: an NDArray now runs through ONNX Runtime models, serves as an ML.NET IDataView and shares its memory with System.Numerics.Tensors.

🧭 Overview

• NumPy 2.x API coverage around 95.2%, everything except datetime, complex64 and other exotic dtypes is fully supported, with byte-to-byte equality and compatibility to NumPy.
  • Text - NumPy's fixed-width bytes_ (S) and str_ (U) dtypes and NEP 55's variable-width StringDType, with every computation NumPy runs on text, all 46 numpy.strings functions and numpy.char with chararray.
  • np.polynomial.* - the whole numpy.polynomial package, all 193 functions and the six series classes for all six bases, bit-exact.
  • np.* - +48 functions, the ten most used on GitHub: np.hypot, np.histogram, np.logspace, np.gradient, np.vectorize, np.hanning, np.geomspace, np.apply_along_axis, np.trapezoid and np.histogram2d.
  • np.random.* - every NumPy bit generator, the whole Generator distribution surface, and a legacy RandomState that is NumPy's own sampler code line for line, all byte-identical streams.
  • np.ma.* - the masked-array module; np.emath.* - the automatic-domain math module; np.matrix - numpy.matrix as the new NDMatrix type.
• DType is NumPy's two-level dtype model - NDArray.dtype returns a numpy.dtype-shaped descriptor, with np.dtypes, byte order and datetime64 / timedelta64 descriptors, and NumPy's scalar classes (np.integer, np.floating, np.number, ...) with np.typing.
• Reductions on NumPy's own schedules - max, min, ptp, nanmax, nanmin, argmax, the flat sum / mean and the fused np.evaluate reductions match NumPy bit for bit, down to the sign of a ±0 tie and a NaN's payload; mean, var, std and their NaN-aware forms are NumPy's own statements; and most of them run faster.
• NumPy's structure for the engine - every computing np.* function is a TensorEngine member named after its NumPy source file, numpy.linalg and numpy.fft are replaceable sub-engines, and the default engine is settable.
• The 2026-11-02 parity QA - 53 of its 55 findings fixed on the owner's decisions (one kept as a documented divergence, one deferred), and NumPy's own dot functions behind np.dot / np.vdot / np.vecdot.
• 86 NumPy-parity, memory-safety and tooling fixes, 40 measured speedups and 44 breaking changes, across the release's 1,670 commits.
• 3 new NuGet packages - NumSharp.Interop.System.Numerics.Tensors, NumSharp.Interop.OnnxRuntime and NumSharp.Interop.MLNet.
• 1,119,305 NumPy 2.4.2 oracle test cases (0.70.0: 119,517) and 20,506 test methods (0.70.0: 14,416).

📦 New NuGet Packages

• NumSharp.Interop.System.Numerics.Tensors - zero-copy interop between NDArray and the BCL's Tensor<T> / TensorSpan<T> (System.Numerics.Tensors), all 15 dtypes.
• NumSharp.Interop.OnnxRuntime - run ONNX Runtime models on NDArrays, fed from the arrays' own memory, with the outputs read back as NDArrays.
• NumSharp.Interop.MLNet - an NDArray as an ML.NET IDataView, and a transformer's output column back as an NDArray.

Full NumSharp 0.80.0 release notes