Dmon.Providers.LlamaCpp 0.2.0

Dmon.Providers.LlamaCpp

Managed local llama.cpp provider for the dmon coding agent.

This extension spawns and owns a llama-server subprocess, exposes it as a Microsoft.Extensions.AI IChatClient, and gates tool-calling capability via a startup probe — so the agent knows at session start whether the chosen model and its chat template actually support tool calls.

Prerequisites

llama-server must be on your PATH. This package does not bundle or download the binary.

The first run fetches the GGUF from Hugging Face via llama-server -hf <owner/repo>. Subsequent runs use the local cache (~/.cache/llama.cpp or HF_HOME). Ensure outbound internet access on first use.

Usage

One-liner (coming in a future release)

Not yet available. The method below depends on DmonHostBuilder.AddProvider(IProviderExtension) being exposed by dmon-core. That hook is planned for a future release. Once it lands, registration will collapse to:

// FUTURE — not yet implemented
await DmonHost.CreateBuilder(args)
    .UseLlamaCpp("unsloth/gemma-4-26B-A4B-it-qat-GGUF")
    .Build()
    .RunAsync();

Interim usage (works today)

Until the dmon-core hosting API is available you can register the extension manually. This path is illustrative and not yet covered by automated integration tests; the exact API surface is subject to change when the hosting hook lands.

var options = new LlamaCppOptions
{
    ModelId = "unsloth/gemma-4-26B-A4B-it-qat-GGUF",
    Quant   = "Q4_K_M",        // optional — default is Q4_K_M
    GpuLayers = 35,            // optional — omit to use CPU only
};

var ext = new LlamaCppProviderExtension(options);

// Resolve IProviderRegistry from the dmon host's DI container
IProviderRegistry registry = host.Services.GetRequiredService<IProviderRegistry>();
await registry.RegisterExtensionAsync(ext);

// Then select llamacpp as the active provider/model via the dmon session API

Options

Configure via LlamaCppOptions or set environment variables before starting the host.

LlamaCppOptions.FromEnvironment() constructs an options record from the env vars in the table above.

Tool-calling behaviour

llama-server is always launched with --jinja to enable Jinja2 chat-template rendering, which is required for tool-call formatting.

After the server reaches readiness, the extension runs a startup probe: it sends a single tool-call request and checks whether the response includes a properly formed tool invocation. The outcome is stored in LlamaCppRuntimeState.ToolCallingVerified and surfaced via ChatClientCapabilities.SupportsToolCalling — both through the CapabilitiesDecorator and on the ModelInfo entries returned by ListModelsAsync.

• If the probe round-trips successfully: SupportsToolCalling = true.
• If the model or its chat template cannot produce tool calls: SupportsToolCalling = false and a warning is emitted. Tool calling is disabled for the session — the agent falls back to plain text.

The probe never guesses capability from the model name or GGUF filename. The result is authoritative for the session.

Building from the monorepo (contributors)

Dmon.Providers.LlamaCpp lives under providers/ in the dmon-core monorepo. Dmon.Abstractions is consumed via ProjectReference — no local feed or manual pack step is required.

# build / test the whole solution (recommended)
dotnet build Everything.slnx -c Release
dotnet test  Everything.slnx -c Release

# or scope to the providers solution
dotnet build providers/providers.slnx -c Release
dotnet test  providers/providers.slnx -c Release

# pack the provider package
dotnet pack providers/Dmon.Providers.LlamaCpp/Dmon.Providers.LlamaCpp.csproj -c Release -o ./artifacts

When consuming the published NuGet package, Dmon.Abstractions arrives as a normal transitive package dependency — no manual feed configuration is needed.