SpawnDev.AI.Server 1.4.3

SpawnDev.AI

💜 Built and maintained by one independent developer — no company, no overhead, just code. If SpawnDev.AI saves you time, please consider sponsoring its development ». Sponsorship is what keeps it alive and maintained.

Run and serve local LLMs and image generation everywhere .NET runs - desktop and the browser - on the SpawnDev.ILGPU.ML inference engine (GGUF models, KV-cache decode, SD-Turbo image generation, WebGPU dispatch-plan capture/replay). LLM decode and SD-Turbo text-to-image both run in-browser on WebGPU, no server or native install.

Why

Local LLM serving shouldn't require a native install. The same AiApiRouter that answers curl localhost:11434/api/chat on a desktop can run inside a shared worker in a browser tab, decoding on WebGPU at interactive speed (qwen3:4b decodes at ~65 tok/s in the browser on an RTX 4070). On the desktop the batch decoder serves llama3.1:8b at 45.5 tok/s, with the first token 204 ms after 200 new tokens over a cached 3.7k-token prompt. Tool calling is parsed server-side into structured calls on every protocol surface.

Tools & in-browser reliability

The chat model can generate images and answer questions about the SpawnDev libraries and crew via a server-side tool system (one registration serves the internal agentic loop, MCP, and protocol clients; binary outputs travel out of band through an artifact store). Because the in-browser default is a tiny model, the engine makes the common cases reliable without depending on the model's own tool-calling: it pre-emptively forces the image tool on a clear "draw X" request and grounds SpawnDev questions from a daily-built repository digest fetched once from a CDN. Details: Docs/reliability.md.

Documentation

Docs/ - protocols (every endpoint + client compatibility), engines (chat, memory, faces, speech, voice, LoRA adapters, model install), hosting (desktop + browser + the demo), tools, and reliability.

Quick start (desktop, Ollama-compatible)

using SpawnDev.AI.Server;

var store = new OllamaModelStore();                    // ~/.ollama/models (or OLLAMA_MODELS)
var registry = new ModelRegistry(new OllamaCacheModelProvider(store), accelerator);  // any SpawnDev.ILGPU accelerator
var engine = new AiChatEngine(registry);
var router = new AiApiRouter(engine);
// host it: map every request to router.TryHandleAsync(method, path, bodyJson, yourTransport)

The demo: a room of characters

Two hosted builds of the same Blazor WASM app:

Both run the group chat entirely in your browser on WebGPU. You make characters - name, persona, model, voice, body and how animated they are - and they talk to each other and to you. Everything persists in OPFS, so a cast survives a reload.

• A scene belongs to the room, not to a character, and every member is handed it verbatim. A persona is who someone IS; a scene is where everybody is right now.
• Each member hears the ones before it in the same round, which is what makes it a conversation rather than several models answering the same question.
• A voice per character, streamed sentence-by-sentence so it starts talking after the first sentence rather than the whole reply, with a stop button.
• A body. Physical actions written in asterisks (*tilts head*) drive an on-screen Reachy-shaped avatar - or your real Reachy Mini when connected. Both perform the same Gesture vocabulary, so a scene reads the same either way. There is one robot, so if two characters ask for it the first keeps it and the rest are drawn.
• Models are opt-in. Every model states its size and what it is for, and nothing large downloads until you press the button that says so. Eight are offered, from a 369 MB starter to gemma4:12b.

Connecting a Reachy Mini

On the Hugging Face Space, open Group chat, then under Reachy Mini press Connect my Reachy. That signs you into Hugging Face (if needed) and reaches your wireless robot through HF's signalling server - the one registered to the account you signed in with. The layout around that control is still rough; the path itself is the one that works.

⚠️ The GitHub Pages build has no working Reachy connect: it is fine for chat, voice, and the on-screen avatar, but not for driving the physical robot.

⚠️ A local http://localhost run can still talk to a robot's LAN daemon directly. An HTTPS page cannot (mixed content) - that is why the hosted path uses WebRTC signalling instead.

Status

Stable since 1.1.0. Extracted from the proven SpawnDev.ILGPU.ML Ollama-server example and verified against the Claude CLI, Ollama clients, and OpenAI-compat clients. The desktop HTTP host, the browser shared-worker host (LLM chat + SD-Turbo image generation on WebGPU), the agentic tool loop, MCP, and the live demos (Hugging Face Space for Reachy, GitHub Pages for everything else) all ship today.

The SpawnDev Crew

• LostBeard (Todd Tanner) - Captain, library author, keeper of the vision
• Riker (Claude CLI #1) - First Officer, implementation lead on consuming projects
• Data (Claude CLI #2) - Operations Officer, deep-library work, test rigor, root-cause analysis
• Tuvok (Claude CLI #3) - Security/Research Officer, design planning, documentation, code review
• Geordi (Claude CLI #4) - Chief Engineer, library internals, GPU kernels, backend work
• Seven (Claude CLI #5) - Wasm backend, GPU kernels, fail-loud verification

🖖