Starlings.Models.EmbeddingGemma2 0.3.0

Starlings.Models.EmbeddingGemma2

Embeddings from google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b, on ONNX Runtime 1.30.0, in-process and offline after an explicit download. The package is versioned 0.x, so the API may still change between minor versions; maturity is per profile (below).

• Text (EmbeddingGemma2TextEmbedder, EmbeddingGemma2Encoder): owned tokenizer or token ids, on the released text graph. The text encoder rejects the ids of the media placeholder and boundary tokens (<|image|>, <|audio|>, <|video|>, <|image>, <image|>, <|audio>, <audio|>).
• Images (EmbeddingGemma2MultimodalEmbedder): an image alone, or text and images interleaved, from already-decoded RGB (EmbeddingGemma2Image), default budget of 280 soft tokens per image. See Images.
• Videos (EmbeddingGemma2MultimodalEmbedder, EmbeddingGemma2Video): a video alone, or text, images and videos interleaved, from already-decoded RGB frames, at the checkpoint's settings (1 frame per second, at most 32 frames, 140 soft tokens per frame, no timestamps). Audio is not supported. See Videos.
• Right-padded batching, mean pooling over every attended token (prompt and image tokens included), L2 normalization: 768-dimensional unit vectors, or a Matryoshka size of 512, 256 or 128 (truncate, then normalize again, as sentence-transformers truncate_dim). Other sizes are rejected.
• Prompts are explicit and never added for you: EmbeddingGemma2Prompt.Query ("task: search result | query: ") for queries, EmbeddingGemma2Prompt.Document ("title: none | text: ") for documents, any of the 20 names in the model's config_sentence_transformers.json through EmbeddingGemma2Prompt.FromName, or None (the default). The model card: "Prefixes apply to text only. Pass images, video, and audio without any prefix."
• Inputs above 8,192 tokens (the card's context window; special tokens, prompt and image tokens included) are rejected with EmbeddingGemma2InputTooLongException, never truncated.
• CUDA runs with TF32 disabled (use_tf32=0); with TF32 the text embeddings drift by up to 1.9e-4 and the image embeddings by up to 1.4e-3.
• The package ships the asset locks (repository, commit, sizes and SHA-256), not the model files (1.1 GB for text, 2.9 GB with images). Acquisition is an explicit call; inference never downloads.

Maturity

Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.

Tokenizer

Text goes through the owned tokenizer Starlings.Tokenizers (profile gemma-bpe-embeddinggemma2, exact against huggingface/tokenizers on the published fixtures): EmbeddingGemma2Tokenizer adds <bos> and <eos> and never truncates. No Microsoft.ML.Tokenizers and no native tokenizer library. Any other IEmbeddingGemma2Tokenizer must reproduce the pinned tokenizer.json exactly.

Install

dotnet add package Starlings.Models.EmbeddingGemma2
dotnet add package Microsoft.ML.OnnxRuntime --version 1.30.0          # CPU natives
# or, for CUDA on Linux x64: Microsoft.ML.OnnxRuntime.Gpu.Linux 1.30.0 (reference exactly one)
# optional, to decode image files: Starlings.Models.EmbeddingGemma2.Imaging.SkiaSharp
# optional, to decode video files on your own FFmpeg 8.x: Starlings.Models.EmbeddingGemma2.Video.FFmpeg
#   (or on FFmpeg 9.x: Starlings.Models.EmbeddingGemma2.Video.FFmpeg9)

Use

using Starlings.Models.EmbeddingGemma2;
using Starlings.Runtime.Onnx;

await EmbeddingGemma2ModelAssets.DownloadAsync();                       // once, explicit; verifies SHA-256
EmbeddingGemma2ModelFiles files = EmbeddingGemma2ModelAssets.OpenLocal(); // offline, verified, never downloads

using var encoder = EmbeddingGemma2Encoder.Create(files.ModelPath, new EmbeddingGemma2EncoderOptions
{
    Session = new OnnxSessionOptions { ExpectedModelSha256 = EmbeddingGemma2ModelAssets.ModelSha256 },
    Dimension = 768,                                                    // or 512, 256, 128
});

var tokenizer = EmbeddingGemma2Tokenizer.FromFile(files.TokenizerPath);   // verifies the pinned SHA-256
var embedder = new EmbeddingGemma2TextEmbedder(tokenizer, encoder);
float[] q = embedder.Embed("how much protein should a female eat", EmbeddingGemma2Prompt.Query);
float[][] docs = embedder.EmbedBatch(["As a general guideline, ..."], EmbeddingGemma2Prompt.Document);

Vectors are normalized, so cosine similarity is the dot product.

Batching options

EmbedBatch splits a batch into ONNX Runtime calls, each right-padded to its longest row. EmbeddingGemma2EncoderOptions controls the split:

The defaults are the qualified behavior: chunks are consecutive rows in input order, with outputs unchanged bit for bit. With SortByLength, which rows share a call depends on the whole batch, so a row's embedding can differ from the default by up to the profile's padded-batch-vs-single bound (CPU tests: at most 1.5e-7 against 5e-6 on 46 fixture rows of 2 to 1,743 tokens in random orders). If a call is cancelled, the Span<float> overload may already hold the rows of the chunks that completed: a prefix of the rows by default, the shortest rows with SortByLength.

DownloadAsync fetches the tokenizer and prompt configuration from google/embeddinggemma-2 and the text graph from the public Hugging Face repository Starlinghq/embeddinggemma-2-onnx at a pinned commit (EmbeddingGemma2ModelAssets.ModelSource), with no token. To use a graph you produced yourself with model-pipeline/reference/embeddinggemma2/release_embeddinggemma2_onnx.py (the digest must match), use OpenDirectory.

Images

await EmbeddingGemma2ModelAssets.DownloadImageAssetsAsync();            // text files + vision graph + multimodal text graph
var imageFiles = EmbeddingGemma2ModelAssets.OpenLocalImageAssets();
using var images = EmbeddingGemma2MultimodalEmbedder.Create(imageFiles);  // SHA-256 of each graph checked while loading

// Decoded RGB from your own stack (3 bytes per pixel, rows top to bottom, no padding), or the SkiaSharp adapter:
EmbeddingGemma2Image photo = EmbeddingGemma2Image.FromRgb24(width, height, rgbBytes);
float[] e = images.Embed(photo);                                          // an image alone, no prompt (the card)
float[] mixed = images.Embed(EmbeddingGemma2MultimodalInput.Create("A cat resting on a blanket. ", photo));
float[] text = images.Embed(EmbeddingGemma2MultimodalInput.FromText("a photo of a cat"), EmbeddingGemma2Prompt.Query);

• Pipeline (upstream, restated from decoded RGB). Aspect-preserving target size with sides that are multiples of 48 and at most 2,520 patches; PyTorch's uint8 antialiased bicubic resize, ported bit for bit (EmbeddingGemma2ImageProcessor); x float32(1/255); 16x16 patches with (x, y) positions; the vision graph (16 layers, 3x3 pooling, projection to 512); the soft tokens replace the token embeddings of <|image> + n x <|image|> + <image|> in the multimodal text graph; mean pooling over every token.
• Interleaving. EmbeddingGemma2MultimodalInput.Create(parts) takes strings and images in order; the text parts are concatenated with no separator and each image is expanded at its position. Text parts must not contain media placeholders. sentence-transformers' {"text": "... <|image|>", "image": img} becomes Create("... ", img); text without a placeholder followed by images becomes Create(text, img).
• Graphs. Inputs with images run on the vision graph and the multimodal text graph (release embeddinggemma-2-vision-onnx-r1); text-only inputs run on the released text graph and equal EmbeddingGemma2TextEmbedder bit for bit (it loads on the first text-only input). The multimodal graph also reproduces the released graph bit for bit on text-only input.
• Limits. Images above 50 megapixels (MaxImagePixels) throw EmbeddingGemma2ImageTooLargeException; inputs above 8,192 tokens after image expansion (about 29 images at 280 tokens) throw EmbeddingGemma2InputTooLongException. Both are checked before any inference.
• Upstream conversion rules apply to whatever decoder you use: alpha dropped (not composited), grayscale replicated, palettes looked up, EXIF orientation not applied, 8 bits per channel (16-bit images are rejected by the SkiaSharp adapter; upstream turns them white).
• Budget. 280 soft tokens per image (the checkpoint's default); MaxImageSoftTokens also accepts 70, 140, 560 and 1,120 (experimental: proposed rows from sentence-transformers, within the protected bounds).
• Memory and batching. One image per vision-graph call by default (MaxVisionBatchSize = 1): on ONNX Runtime CUDA one 2,520-patch image needs about 610 MB of attention buffer per layer, and two did not fit in 5.5 GB. On CPU, vision batches equal single images bit for bit; raise MaxVisionBatchSize and MaxVisionBatchPatches to trade memory for throughput. Each call is padded to its longest image (PadImagesToMaxPatches = false; upstream pads every image to 2,520 patches, which gives the same soft tokens on CPU and within 3.4e-5 on CUDA).
• CUDA. The vision graph needs cuDNN 9 on the loader path (its ReduceMax runs through cuDNN); the three graphs take about 2.9 GB of GPU memory.
• Speed (indicative, i7-5820K 12 threads; RTX 4060 Ti). Vision about 4.5 s per full-budget image on CPU, 0.23 s on CUDA; preprocessing about 1.5 to 2 ms for a photo and 5 ms for 4000x3000. The resize and the patch values run on AVX2 when the CPU has it and split the rows of large images over the thread pool; the bytes and values are the same on every path (integer resize, one float32 product per value).

Videos

var imageFiles = EmbeddingGemma2ModelAssets.OpenLocalImageAssets();        // same graphs as images
using var embedder = EmbeddingGemma2MultimodalEmbedder.Create(imageFiles);

// Every decoded frame (RGB24) plus the average frame rate and duration: the package samples as upstream.
EmbeddingGemma2Video video = EmbeddingGemma2Video.FromDecodedFrames(frames, framesPerSecond: 30, durationSeconds: 5);
float[] e = embedder.Embed(video);                                          // a video alone, no prompt (the card)
float[] mixed = embedder.Embed(EmbeddingGemma2MultimodalInput.Create("The clip ", video, " shows moving shapes."));

// Or decode with the optional adapter on your own FFmpeg 8.x (Starlings.Models.EmbeddingGemma2.Video.FFmpeg):
// var clip = new EmbeddingGemma2VideoDecoder().DecodeFile("clip.mp4"); embedder.Embed(clip.Video);

• Frames in, three forms. FromDecodedFrames(frames, fps, duration): every decoded frame with the timing torchcodec reports (packet count, max(pts + duration) - min(pts), frames / duration); only the sampled frames are read, so frames may be a lazy list. FromSampledFrames(frames): 1 to 32 frames you sampled yourself, used as given (equal to the file only when they are upstream's frames). FromFramesWithoutTiming(frames): rejected by default with EmbeddingGemma2VideoTimingRequiredException; AllowVideosWithoutTiming = true opts in to upstream's behaviour for an array without metadata (every frame, thinned to 32), which gives a different embedding than the same file (cosine 0.965 on a 5 s fixture clip).
• Sampling (upstream, exact). EmbeddingGemma2VideoSampling.SampleIndices: max(1, int(duration)) candidates at 1 frame per second, candidate i = frame min(total - 1, int(i * fps)), more than 32 thinned like np.linspace(..., dtype=int). Quirks kept for parity: a clip under 2 s gives one frame; sampling is by frame index from the average rate (variable frame rate samples frames, not seconds).
• Per frame. The image preprocessing at 140 soft tokens (1,260 patches; same resize kernel; up to 8 frames preprocessed in parallel ahead of their vision calls, EmbeddingGemma2ImageProcessor.Preprocess(frames, 140) for your own pipeline), the vision graph in calls of at most MaxVideoFramesPerVisionCall frames (default 2, GPU-safe; on CPU frame batches equal single frames bit for bit), and <|image> + n x <|video|> + <image|> per frame with no separator and no timestamps. A video costs frames x (n + 2) tokens, at most 4,544; inputs above 8,192 tokens are rejected before inference (two maximal videos do not fit).
• Decoding. FFmpeg's default conversion is what upstream embeds (10-bit and odd sizes included); other converters move the embedding by up to 1.7e-2. Use the FFmpeg adapter (exact against torchcodec on the fixtures) or an equivalent FFmpeg call (ffmpeg -i in.mp4 -f rawvideo -pix_fmt rgb24 - plus torchcodec-style metadata).
• Speed (indicative). Vision about 2.7 s per frame on CPU (i7-5820K, 12 threads: a 32-frame video takes about 70 to 100 s) and about 0.07 s on CUDA (RTX 4060 Ti); prefer CUDA for video.

Memory on Linux

These apply to every profile on Linux.

• Heap trim on release (on by default). When a session's native memory is released (after Dispose and the last in-flight run), Starlings.Runtime.Onnx calls glibc's malloc_trim(0), so a process that loads and unloads the model returns the freed memory instead of keeping it resident (text on the hosted runner: about 1,480 MiB after unloading without it, 400 to 500 MiB with it). It changes no allocator setting. Opt out per session with OnnxSessionOptions.TrimNativeHeapOnRelease = false, or for the whole process with the AppContext switch Starlings.Runtime.Onnx.DisableNativeHeapTrim (details in the Starlings.Runtime.Onnx README).
• No CPU memory arena on CPU (default). The package creates its CPU sessions with ONNX Runtime's CPU memory arena off (OnnxSessionOptions.CpuMemoryArena, null by default, means off here; set it to true to restore the arena). With the arena, the loaded video embedder's resident memory rose by about 2.2 GB in 15 minutes on the hosted runner with one session kept loaded, and the arena's freed regions stayed resident in glibc's per-thread heaps after unloading; without it, the loaded footprint is about a third smaller (video about 2.7 GB instead of 3.9 GB, text 1.8 GB instead of 2.1 GB) and the one-hour hosted soaks of text and video passed in three qualification runs out of three (they failed before). Embeddings are bit-identical with and without the arena (a test checks it); warm latency changed by 1 to 2% on text and not measurably on video (development runs). Evidence: EmbeddingGemma 2 hosted soak.
• Dedicated run threads on CPU (default). The package's CPU sessions run on the runtime's dedicated run threads (OnnxSessionOptions.DedicatedRunThreads, null by default, means on here; set it to false to run on the calling thread). Without the arena, each run's buffers are freed into the calling thread's glibc heap, which keeps up to 64 MiB of them resident; callers from the .NET thread pool rotate, and the hosted one-hour soaks of text and images failed intermittently as more and more heaps kept that memory (text: 3 to 7 heaps of about 19 MiB; images: 2 to 8 heaps of up to 59 MiB). On a fixed set of threads (at most one per processor, started only when all are busy) the same heaps are reused. Embeddings are bit-identical either way (a test checks it). Evidence: EmbeddingGemma 2 soak reliability.
• Heap trims after load and after each run on the image and video path (CPU, default). EmbeddingGemma2MultimodalEmbedder creates its CPU sessions with OnnxSessionOptions.TrimNativeHeapAfterLoad and TrimNativeHeapAfterRun on (null by default means on here; set them to false to turn them off). The text encoder does not use them. Without them, the loaded image embedder kept about 400 MiB freed during session creation in the main glibc heap, and 60 to 190 MiB of freed run buffers per run thread, varying from load to load and rising in steps over an hour. With them, the loaded footprint on the hosted runner is about 2.05 GB instead of 2.7 GB. Embeddings are bit-identical either way (a test checks it), and image request latency was unchanged on the hosted runner (p50 18.0 to 18.1 s with them, 18.0 to 18.3 s without, EPYC 7763). Evidence: EmbeddingGemma 2 image loaded soak.
• Reused request buffers on the image and video path. EmbeddingGemma2MultimodalEmbedder keeps the large buffers of a request (resized pixels, patch values, the vision and multimodal graph inputs and outputs, soft tokens, resampler weight tables) in a pool it owns and reuses them for the next request; it releases them on Dispose. Each request rents its buffers for its own use, so concurrent callers never share one, and the pool keeps only what concurrent requests used at the same time. Before, every image request allocated and freed several arrays on the .NET large object heap, which is not compacted, and its fragmentation moved the process's committed memory by 60 to 130 MiB over an hour. Managed allocation per request fell from about 11 MiB to 0.4 MiB, and peak resident memory for one caller was lower (2,871 MiB instead of 3,099 MiB in the same measurement). Embeddings are bit-identical (tests compare single, batched and concurrent requests with and without the pool, and the outputs with the code before the pool). Evidence: EmbeddingGemma 2 image loaded soak.
• Transparent huge pages. The qualification soaks ran with /sys/kernel/mm/transparent_hugepage/enabled set to madvise (the Ubuntu default). On hosts set to always, the kernel's khugepaged fills in partly used 2 MiB heap ranges, so resident memory can look larger and rise over time without any leak. Evidence: scheduled regression fix.
• The Stable CPU profiles (onnx-fp32-cpu, image-fp32-cpu, video-fp32-cpu) and the Stable CUDA text profile (onnx-fp32-cuda) need no MALLOC_* setting; their one-hour soaks ran with the defaults. The CUDA video profile (video-fp32-cuda) requires MALLOC_MMAP_THRESHOLD_=65536 (below).

Image and video on CUDA

Deployment requirement for embeddinggemma-2/video-fp32-cuda: set MALLOC_MMAP_THRESHOLD_=65536 in the environment of a Linux process that runs it. It is recommended for image-fp32-cuda too.

With default glibc settings, a process that loads and unloads the multimodal embedder repeatedly keeps about 0.7 to 0.8 GiB of freed native memory resident in glibc's main heap between loads, and that high-water mark creeps upward with each new peak (allocator retention, not a leak: malloc_trim returns it and the trimmed series is flat). In 12-minute CUDA soaks with the thread fix below, the default video run failed the soak's growth rule on the loaded series (316 MiB/h fitted); with MALLOC_MMAP_THRESHOLD_=65536 the image and video footprints were about half as large and flat. Device memory is returned at every unload, and the cuDNN, cuBLAS and driver mappings do not grow.

In a container whose CPU quota is below the host's CPU count, the session now uses .NET's container-aware processor count for its intra-op threads when none is set (ONNX Runtime's own default started one thread, and one malloc arena, per physical core of the host). Evidence: CUDA soak growth.

Status

Every profile of this package is Stable. On Linux x64 CPU: text (embeddinggemma-2/onnx-fp32-cpu) since 2026-10-15, images (image-fp32-cpu) since 2026-10-16 and videos (video-fp32-cpu) since 2026-10-17. On Linux x64 CUDA with an NVIDIA GeForce RTX 3090: text (onnx-fp32-cuda) and images (image-fp32-cuda) since 2026-10-16 and videos (video-fp32-cuda) since 2026-10-17. Each passed every qualification gate, G0 to G7. The weekly qualification workflow reruns the CPU profiles' G1 to G3 on the release sets and a one-hour soak. For text, G2 reads the reference embeddings committed under model-pipeline/protected/. For images and videos, G2 reads the accepted reference tensors hosted at Starlinghq/starling-release-references. Those are pinned by commit and SHA-256 in the release set's protected hosted-reference.json, and every array is checked against the set's tensors.json. GPU regression runs for every release (an ai-v* tag) and on demand, on an RTX 3090 only, in the qualification workflow's cuda (runpod) job: it reruns the CUDA tests, G1 to G3 on the release sets and a soak smoke test.

Videos were the last profiles promoted. Two blockers were fixed first. The hosted one-hour CPU video soak failed the growth rule in every run up to 2026-10-16 (about 280 MB/h). It has passed in every run since the library changes described under Memory on Linux: no CPU arena, dedicated run threads, heap trims after load and after each run, and reused request buffers (evidence). The release-set G2 failed on AVX-512 runners (token embeddings 1.83e-3 against 1e-3). The cause is the vision graph's Softmax kernels in ONNX Runtime (#89), and the PyTorch reference itself moves by up to 2.09e-3 between CPUs. The maintainer recalibrated that one intermediate bound to 4e-3 from the reference's own cross-CPU spread (#90); the final-embedding bound is unchanged. On CUDA, the first per-release run landed on an RTX 4090, outside the contract, and failed video G2 on the final embedding (1.009e-5 against 1e-5). The RTX 3090 runs pass (runs 39012176735 and 39023915026; evidence). Gate evidence: the catalog.

Text (onnx-fp32-cpu): release sets of 232 tokenizer rows (ids exact) and 696 embedding rows (no prompt, query and document; within 5.1e-7 of the PyTorch reference, binding bound 5e-6), MIRACL nDCG@10 0.8325; development fixtures of 138 rows up to 1,750 tokens (within 3.7e-7).

Images (image-fp32-cpu and image-fp32-cuda stable), Linux x64, from decoded RGB (42 protected rows: 28 images alone, 6 interleaved or prompted, 8 text queries): rendered ids exact; resized bytes, pixel values and positions bit-exact on the 28 supported images; embeddings within 2.3e-6 (CPU) and 1.0e-6 (CUDA) of the PyTorch reference, binding bound 1e-5, min cosine 0.9999995. Audio: unsupported. EmbeddingGemma2Capabilities.All lists every capability with its status.

Videos (video-fp32-cpu and video-fp32-cuda stable), Linux x64, from decoded RGB frames (35 protected rows: 22 videos alone, 6 interleaved or prompted, 2 input forms, 5 text queries; synthetic H.264, HEVC, VP9 and AV1 clips of 0.5 to 40 s): sampled indices exact (and on 20,849 upstream cases), rendered ids exact; decoded frames (FFmpeg adapter), resized frames, pixel values and positions bit-exact on all 32 video inputs; embeddings within 1.7e-6 (CPU) and 3.3e-6 (CUDA) of the PyTorch reference, binding bound 1e-5, min cosine 0.9999995. Frames without timing: rejected by default (upstream behaviour on opt-in reproduces its row within 1.2e-7). Timestamps, other sampling settings and audio tracks: unsupported.

Stable contract

This is the contract of profile embeddinggemma-2/onnx-fp32-cpu (text embeddings). Its Stable label (since 2026-10-15) commits to it under the support policy. The statuses are the registry manifest's and EmbeddingGemma2Capabilities.Profiles, which a test keeps equal key for key. Images and videos are the profiles of the contracts that follow.

Stable contract: images

This is the contract of profile embeddinggemma-2/image-fp32-cpu (image and interleaved text-and-image embeddings). Its Stable label (since 2026-10-16) commits to it under the support policy. The statuses are the registry manifest's and EmbeddingGemma2Capabilities.Profiles, which a test keeps equal key for key.

Stable contract: videos

This is the contract of profile embeddinggemma-2/video-fp32-cpu (video and interleaved text, image and video embeddings). Its Stable label (since 2026-10-17) commits to it under the support policy. The statuses are the registry manifest's and EmbeddingGemma2Capabilities.Profiles, which a test keeps equal key for key.

Stable contract: CUDA

These are the contracts of profiles embeddinggemma-2/onnx-fp32-cuda (text) and embeddinggemma-2/image-fp32-cuda (images), Stable since 2026-10-16, and embeddinggemma-2/video-fp32-cuda (videos), Stable since 2026-10-17. Each is its CPU contract above, with these differences. The statuses are the registry manifests' and EmbeddingGemma2Capabilities.Profiles, which a test keeps equal key for key.

Evidence: binding CUDA G4 and G5 round, per-release GPU regression, run 38857860186, per-release GPU regression for videos, runs 39012176735 and 39023915026, release-set gates for text, images and videos, CUDA cancellation.

Qualification evidence

• G0 provenance: the 2026-10-14 provenance check.
• G1, G2 and G3 on the P-030 release sets: text, images, videos.
• G4 one-hour soak: text, 2026-10-14; images and videos in the 2026-10-15 round below. G5 matched benchmark: 2026-10-15.
• G6 distribution and G7 integration: G6, G7.
• Regression support: the weekly qualification workflow (jobs release-gates and soak): text green in run 38822625245; images and text green (release gates and soak) in run 38848321624; videos green (release gates, both FFmpeg decoder checks, and the one-hour soak) in run 39023915026 (evidence). CUDA: per-release GPU regression run 38857860186 (the qualification workflow's cuda (runpod) job).

License and usage policy

The package is MIT OR Apache-2.0; it includes a port of PyTorch's resize kernel (BSD-3-Clause, THIRD-PARTY-NOTICES.md). The model weights, and Starling's ONNX graphs derived from them, are licensed under the Apache License 2.0 by Google DeepMind (model card). The model card also states:

Deployments must adhere to the Gemma Prohibited Use Policy.

The text graph contains only the text path (embedding, per-layer embeddings, 24 layers, final norm, 512-to-768 projection); the vision graph only the vision tower, pooling and embed_vision; the multimodal text graph is the text graph with two inputs for soft tokens. All are converted to float32 and exported to ONNX; the release notices list the modifications.