SharpInference.ModelHandler 1.0.0-alpha.2

SharpInference

A pure C#/.NET AI inference engine for image generation, speech, vision, video, and interactive world models — no Python, no native runtime DLLs.

SharpInference loads .safetensors and .gguf checkpoints directly and runs them on NVIDIA CUDA, cross-vendor Vulkan, or CPU SIMD — entirely in managed .NET. GPU kernels are PTX/SPIR-V shipped with the package and JIT-compiled at runtime; there are no C++ wrappers, no bundled native inference library, and no external Python process to manage. Just NuGet packages.

It is the non-LLM companion to dotLLM: together they form a complete AI stack in pure .NET.

⚠️ Alpha software

This is 1.0.0-alpha — an early, fast-moving preview. Use it to experiment, not in production.

• APIs will change without notice between alpha releases. Pin an exact version.
• Model coverage is broad but maturity varies. Many architectures are implemented and load/run end-to-end but are still being validated numerically against their reference implementations. Treat output quality per-model as "verify before you rely on it."
• No support guarantees, no semver stability until 1.0.0.
• The OpenAI-compatible server and CLI are not published as packages in this alpha — they live in the source repository.

Found a bug or a mismatch against a reference? Please open an issue.

Install

One package pulls in the whole stack (all backends + every modality):

dotnet add package SharpInference --prerelease

Or reference only the pieces you need (see Packages):

dotnet add package SharpInference.Audio --prerelease
dotnet add package SharpInference.Cpu   --prerelease

Requires .NET 8 or .NET 10.

Quick start — speech-to-text

The Whisper pipeline downloads a checkpoint from HuggingFace on first use and runs on whichever backend you pass:

using SharpInference.Audio.Pipelines;
using SharpInference.Core.Backends;
using SharpInference.Cpu;          // or SharpInference.Cuda / SharpInference.Vulkan

using WhisperPipeline pipeline = await WhisperPipeline.LoadAsync("openai/whisper-base");
using IBackend backend = new CpuBackend();

var options = new WhisperOptions { Language = "en", WithTimestamps = false };
string text = pipeline.TranscribeWav(backend, "audio.wav", options);

Console.WriteLine(text);

Swap new CpuBackend() for new CudaBackend() or new VulkanBackend() — the pipeline is backend-agnostic. The same LoadAsync / pipeline pattern applies across modalities (StableDiffusion15Pipeline, WanVideoPipeline, KokoroPipeline, …); see the samples in the repo for image, video, and TTS walkthroughs.

What it can do

Checkpoints load directly from .safetensors / .gguf, including quantized weights (GGUF, MXFP4/8, NVFP4, block-scaled).

Coverage is wide because the engine shares a common core (tensors, schedulers, VAEs, text encoders, DSP) across architectures. Per-model numerical validation is ongoing — see the alpha note above.

Design pillars

Packages

Requirements

• .NET 8 or .NET 10 SDK

CUDA backend (NVIDIA, fastest)

• CUDA 12.x runtime
• NVIDIA GPU, compute capability 8.0+ (RTX 30xx/40xx, A100, H100)

Vulkan backend (NVIDIA / AMD / Intel, cross-vendor)

• Vulkan 1.3+ runtime (ships with the GPU vendor driver)
• GPU with FP16 compute (shaderFloat16) — most discrete GPUs from 2019+

CPU backend

• Any x86-64 (AVX2+) or ARM64 (NEON) machine — no GPU required

Links

• Source & docs: https://github.com/kalebbroo/SharpInference
• Issues: https://github.com/kalebbroo/SharpInference/issues
• LLM companion: dotLLM

License

MIT © 2026 kalebbroo