SharpInference 0.7.0

.NET 10.0 This package targets .NET 10.0. The package is compatible with this framework or higher.
There is a newer version of this package available.
See the version list below for details. 
dotnet add package SharpInference --version 0.7.0
                    


                    

                
NuGet\Install-Package SharpInference -Version 0.7.0
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package. 
<PackageReference Include="SharpInference" Version="0.7.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package. 
<PackageVersion Include="SharpInference" Version="0.7.0" />
                    


                            Directory.Packages.props
                        

                    

                
<PackageReference Include="SharpInference" />
                    


                            Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package. 
paket add SharpInference --version 0.7.0
                    


                    

                
#r "nuget: SharpInference, 0.7.0"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package. 
#:package SharpInference@0.7.0
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package. 
#addin nuget:?package=SharpInference&version=0.7.0
                    


                            Install as a Cake Addin
                        

                    

                
#tool nuget:?package=SharpInference&version=0.7.0
                    


                            Install as a Cake Tool

SharpInference

A high-performance LLM inference engine and image generation pipeline for .NET 10. Reads GGUF model files and runs transformer inference on CPU (AVX2/AVX-512 SIMD) or GPU (Vulkan compute shaders / CUDA cuBLAS). Includes Z-Image-Turbo text-to-image and Real-ESRGAN upscaling.

This is the library package. For a command-line tool, install SharpInference.Cli instead.

Install

dotnet add package SharpInference

Quick start

using SharpInference.Core;
using SharpInference.Cpu;
using SharpInference.Engine;

var model = GgufModelLoader.Load("models/SmolLM2-1.7B-Instruct-Q4_K_M.gguf");
var backend = new CpuBackend();
var forward = new ForwardPass(model, backend);
var engine = new InferenceEngine(forward, model.Tokenizer);

await foreach (var token in engine.GenerateAsync("Hello, ", new SamplingParams { Temperature = 0.7f }))
{
    Console.Write(token);
}

For GPU inference, swap CpuBackend for VulkanBackend or CudaBackend, or use HybridForwardPass to offload selected layers.

What's in the package

All 8 SharpInference library assemblies are bundled in one package:

Assembly Purpose
SharpInference.Core GGUF parsing, BPE tokenizer, tensor types, model graph
SharpInference.Cpu CPU backend (AVX2/AVX-512 SIMD, Q4_K_M dequant, optional OpenBLAS)
SharpInference.Vulkan Vulkan compute backend
SharpInference.Cuda CUDA / cuBLAS backend + NVRTC kernels
SharpInference.Engine Forward pass, paged KV cache, samplers, speculative decoding
SharpInference.Diffusion Z-Image-Turbo + FLUX.1 image generation
SharpInference.Pipeline 3-tier VRAM → RAM → NVMe memory hierarchy
SharpInference.TurboQuant 3-bit KV-cache compression

Optional native dependencies

  • OpenBLAS (CPU GEMM acceleration) — auto-detected on PATH, silently skipped if absent.
  • Vulkan drivers — up-to-date GPU drivers (AMD / Intel / NVIDIA). No extra install on Windows.
  • CUDA Toolkit 11.x or 12.x — cublas64_*.dll and cudart64_*.dll on PATH. NVIDIA only.

NativeAOT

All assemblies are trim-safe and NativeAOT-compatible. To publish a single-binary application:

dotnet publish -c Release -r win-x64

License

MIT. Copyright (c) 2026 Pekka Heikura.

Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on SharpInference:

Package Downloads
SharpInference.Server

ASP.NET Core endpoints, options, and DI extensions that expose SharpInference as OpenAI- and Anthropic-compatible HTTP APIs. Bring your own host (Kestrel, IIS, YARP, etc.).

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.7.2-alpha.0.9 0 6/7/2026
0.7.2-alpha.0.8 0 6/7/2026
0.7.2-alpha.0.7 0 6/7/2026
0.7.2-alpha.0.6 14 6/6/2026
0.7.2-alpha.0.5 34 6/6/2026
0.7.2-alpha.0.4 36 6/6/2026
0.7.2-alpha.0.3 32 6/6/2026
0.7.2-alpha.0.2 44 6/5/2026
0.7.2-alpha.0.1 39 6/5/2026
0.7.1 68 6/4/2026
0.7.1-alpha.0.1 39 6/4/2026
0.7.0 69 6/4/2026
0.6.1-alpha.0.9 40 6/4/2026
0.6.1-alpha.0.8 44 6/4/2026
0.6.1-alpha.0.7 40 6/4/2026
0.6.1-alpha.0.6 38 6/4/2026
0.6.1-alpha.0.5 46 6/4/2026
0.6.1-alpha.0.4 42 6/3/2026
0.6.1-alpha.0.3 45 6/3/2026
0.6.1-alpha.0.2 39 6/3/2026
Loading failed