OpenTail.Stingray.Cli 1.0.6

OpenTail.Stingray.Cli

stingray — a command-line tool for local LLM inference and image generation, powered by OpenTail.Stingray. Reads GGUF models and runs them on CPU (AVX2/AVX-512) or GPU (Vulkan / CUDA). No Python, no sidecar process.

Built by opentail.net

Install

dotnet tool install -g OpenTail.Stingray.Cli

Or update:

dotnet tool update -g OpenTail.Stingray.Cli

The command is stingray.

Usage

# Text generation (CPU)
stingray -m models/SmolLM2-1.7B-Instruct-Q4_K_M.gguf -p "Once upon a time" --temp 0.7

# All layers on GPU (Vulkan or CUDA, auto-selected)
stingray -m models/Qwen3-8B-Q4_K_M.gguf -p "Explain mmap" -g -1

# Interactive chat (omit -p to enter chat mode)
stingray -m models/Qwen3-8B-Q4_K_M.gguf

# Inspect before running
stingray list-metadata -m model.gguf
stingray capabilities                     # which model packages are supported
stingray inspect -m models/some-hf-dir    # verdict on a SafeTensors package

# Image generation (Z-Image-Turbo; CUDA or Vulkan)
stingray image \
  -m models/z_image_turbo-Q5_K_M.gguf \
  --vae models/z-image-turbo/vae \
  --qwen-encoder models/Z-Image-AbliteratedV1.Q5_K_M.gguf \
  --qwen-tokenizer models/z-image-turbo/tokenizer/tokenizer.json \
  -p "a serene mountain lake at sunrise" -W 512 -H 512 --steps 4 -o out.png

Coming from llama.cpp

Flag names follow llama-cli where the meaning matches, including single-dash spellings like -ngl, -ctk, -md and -fa, so a command copied from llama.cpp documentation generally just runs.

Where a flag cannot be honoured, it is refused with a named reason rather than silently ignored — -ts, -sm, -mg, --mlock, --numa, --presence-penalty and -b/-ub all tell you what Stingray does instead. Flags that are simply inert here (-fa, --no-warmup) are accepted with a note so a pasted command line still runs. The one thing you will never get is a flag that looks accepted and quietly does nothing.

Common flags

Run stingray --help for the full reference.

Requirements

• .NET 10 runtime — the tool installs framework-dependent
• x86-64 CPU with AVX2
• GPU inference is optional: any Vulkan-capable GPU (AMD / Intel / NVIDIA), or an NVIDIA GPU with CUDA 12.x — the CUDA path is pinned to the 12.x runtime SONAMEs (cudart64_12, cublas64_12)

Links

• Library package
• Server package
• Repository and documentation

Acknowledgements

Forked from SharpInference by Pekka Heikura (MIT), which remains actively developed upstream; its copyright is retained in THIRD_PARTY_NOTICES.md.

Interoperates with llama.cpp's GGUF format and quantization block layouts, and follows llama-cli flag names where the meaning matches — no llama.cpp code is used. LLamaSharp was studied as the reference for .NET inference API design; no LLamaSharp code is used, and unlike it this engine is managed C# end to end rather than P/Invoke bindings to native llama.cpp.

License

MIT. Copyright © 2026 OpenTail.