Starlings.Models.Gliner2 0.3.0

Starlings.Models.Gliner2

English entity extraction with fastino/gliner2-base-v1 at revision f9634218e53580c56edf0de97ca1a7d3f1c2354e (GLiNER2 2.0.0, source 55656fbf), on ONNX Runtime 1.30.0, in-process and offline after an explicit acquisition. You name the entity labels; the model returns labelled spans with offsets and confidences. The package is versioned 0.x, so the API may still change between minor versions; maturity is per profile (below).

• GLiNER2's text handling is ported with Python semantics (normalization, the regular-expression word splitter, str.lower()), and the schema prefix is encoded with the owned tokenizer Starlings.Tokenizers (profile gliner2-base-v1-unigram). No Microsoft.ML.Tokenizers, no native tokenizer, no Python.
• One ONNX graph (token ids and routing positions to count and span logits); GLiNER2's count gate, threshold, overlap suppression and result order in C#.
• The package ships the asset lock (sizes and SHA-256), not the 838 MB of model files. Acquisition is an explicit call; inference never downloads.

Maturity

Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.

Install

dotnet add package Starlings.Models.Gliner2
dotnet add package Microsoft.ML.OnnxRuntime --version 1.30.0          # CPU natives
# or, for CUDA on Linux x64: Microsoft.ML.OnnxRuntime.Gpu.Linux 1.30.0 (reference exactly one)

Use

using Starlings.Models.Gliner2;

await Gliner2ModelAssets.DownloadAsync();                   // once, explicit, needs the network; verifies SHA-256
Gliner2ModelFiles files = Gliner2ModelAssets.OpenLocal();   // offline, verified, never downloads
using var extractor = Gliner2EntityExtractor.Create(files);

var result = extractor.Extract(
    "Apple CEO Tim Cook announced the iPhone 15 in Cupertino.",
    ["company", "person", "product", "location"],
    threshold: 0.5f);
foreach (var e in result.Entities)
{
    Console.WriteLine($"{e.Label}: {e.Text} [{e.Start}, {e.End}) {e.Confidence:F3}");
}

Batching options

ExtractBatch and RunModel split a batch (one label set) into ONNX Runtime calls, each right-padded to its longest text. Gliner2ExtractorOptions controls the split:

The defaults are the qualified behavior, with outputs unchanged bit for bit. With SortByLength, which texts share a call depends on the whole batch, so span logits can differ from the default by up to the profile's padded-batch-vs-single bound (padded_batch_vs_single_span_logits_relative); the CPU tests saw no difference and identical entities on the 54 in-contract fixture cases in random orders. On CUDA, calls stay serialized (SerializeNativeCalls); the options only change how one request is chunked.

Stable contract

This is the contract of profile gliner2-base-v1/onnx-fp32-cpu. Its Stable label commits to it under the support policy. The statuses are the registry manifest's and Gliner2Capabilities.Profiles, which a test keeps equal key for key.

Stable contract: CUDA

This is the contract of profile gliner2-base-v1/onnx-fp32-cuda (Stable since 2026-10-16). It is the CPU contract above, with these differences. The statuses are the registry manifest's and Gliner2Capabilities.Profiles, which a test keeps equal key for key.

Evidence: binding CUDA G4 and G5 round, per-release GPU regression, run 38857860186, release-set gates, CUDA cancellation.

Micro-batching (experimental, opt-in)

Gliner2ExtractorOptions.MicroBatching (null by default: off) applies to ExtractAsync only; Extract, ExtractBatch and the staged methods never queue. Without it, ExtractAsync runs on the calling thread like Extract and returns a completed task. With it, each call's text is prepared and length-checked on the calling thread (an invalid text fails only its own call), then queued; one dispatcher thread runs the queued texts of concurrent callers that share a schema (the same labels in the same order and the same descriptions) in padded batches of at most MaxRows texts and MaxTokens padded token cells, and each caller decodes its spans with its own threshold. MaxBatchSize still splits each batch into graph calls, and the CUDA run gate still applies, so on CUDA serialized single calls become serialized batched calls. Outputs then depend on the concurrent traffic: equal to a single call within the profile's padded-batch tolerance, not bit for bit reproducible. Cancelling a queued call removes it; cancelling a dispatched one cancels only that caller's task (the batch keeps running for the others). A full queue (QueueCapacity) makes callers wait for a slot. Dispose faults queued calls with ObjectDisposedException and waits for a running batch. MicroBatchingStatistics reports batches, items, padding and queue and run time. Status: experimental (capability micro_batching): weight-free tests and one real-model test within the padded-batch tolerance on Linux x64 CPU; no benchmark adoption decision, no CUDA run and no soak yet.

Memory on Linux

Deployment requirement: set MALLOC_MMAP_THRESHOLD_=65536 in the environment of a Linux process that runs this profile (CPU or CUDA).

With default glibc settings, a process that loads and unloads the extractor repeatedly keeps freed native memory resident: about 1.2 to 1.5 GiB between loads, rising by about 65 to 90 MiB over the first half hour of cycling and then holding. This is allocator retention (glibc malloc's resident high-water mark and fragmentation), not a leak: with the threshold set, every unload returns what the load and its requests allocated. But the soak's growth rule, which is not changed for this, judges the default run: the one-hour default soak fit 81 to 82 MiB/h over the whole run and failed the rule, although the curve flattens after about 30 minutes. With MALLOC_MMAP_THRESHOLD_=65536 the footprint after unloading held at about 76 MiB, the loaded footprint was about 740 MiB lower, and the one-hour G4 soak passed.

Evidence: GLiNER2 soak growth (diagnosis, six controlled runs), G4 qualification soak.

Status

Profiles gliner2-base-v1/onnx-fp32-cpu (Linux x64 CPU) and gliner2-base-v1/onnx-fp32-cuda (Linux x64, RTX 3090, Stable since 2026-10-16) have passed every qualification gate, G0 to G7; their maturity is in the table above. GPU regression runs for every release and on demand, on an RTX 3090 only (the qualification workflow's cuda (runpod) job); CPU regression runs weekly. Evidence and reproduction: ai-dotnet/README.md and the catalog. License of the weights and the converted graph: Apache-2.0 (model card); GLiNER2 by Fastino AI, backbone initialised from microsoft/deberta-v3-base (MIT).

Qualification evidence

• G0 provenance: the 2026-10-14 provenance check.
• G1, G2 and G3 on the P-030 release sets: release-set gates.
• G4 one-hour soak with MALLOC_MMAP_THRESHOLD_=65536 and G5 matched benchmark: 2026-10-15.
• G6 distribution and G7 integration: G6, G7.
• CUDA: G4 and G5 in the binding CUDA round; per-release GPU regression run 38857860186.
• Regression support: the weekly qualification workflow (jobs release-gates and soak), and its per-release cuda (runpod) job for CUDA.