InferHub.Client 0.1.0

InferHub.Client

A small, typed .NET client for InferHub — a self-hosted, Ollama-compatible inference mesh.

Point it at a coordinator, pass a Bearer token, and call chat, generate, model listing and status from C# with typed requests, dependency injection, and no heavy dependencies.

v0.1.0 — foundation and core blocking inference. Streaming, embeddings, vector data plane, RAG retrieval and admin follow in later phases.

Install

dotnet add package InferHub.Client

Targets net9.0 and net10.0.

Quick start

using InferHub.Client;
using InferHub.Client.Configuration;
using InferHub.Client.Models.Ollama;
using Microsoft.Extensions.DependencyInjection;

var services = new ServiceCollection();
services.AddInferHubClient(o =>
{
    o.BaseAddress = new Uri("http://localhost:5080");
    o.ApiKey = "<your-client-api-key>";
});

var provider = services.BuildServiceProvider();
var client = provider.GetRequiredService<IInferHubClient>();

var chat = await client.ChatAsync(new ChatRequest
{
    Model = "llama3",
    Messages = new[]
    {
        new ChatMessage { Role = "user", Content = "Say hi in one word." }
    },
    Stream = false
}, CancellationToken.None);

Console.WriteLine(chat.Message?.Content);

What ships in v0.1.0

Request models carry an extension bag (AdditionalProperties), so any unknown fields from the Ollama contract pass through untouched — you can hand-set options, format, tool definitions, etc. without waiting on the client to type them.

Auth

• Non-loopback calls need ApiKey (attached as Authorization: Bearer <key> by a DelegatingHandler).
• Loopback calls to the coordinator skip auth by default (unless the coordinator sets Auth:RequireAuthForLoopback=true).
• /health is always open.
• Admin routes require a separate AdminApiKey and land on a dedicated interface in a later phase; a client key alone never surfaces admin methods.

Errors

Any non-success HTTP response is surfaced as InferHubException, carrying:

• StatusCode — the HTTP status
• Message — the coordinator's { "error": "…" } body if present

The client treats 404 (model missing) and 424 Failed Dependency (retrieval unavailable, added in a later phase) as distinct signals worth checking with StatusCode.

Links

• InferHub server: https://github.com/Dev-Art-Solutions/InferHub
• Product page: https://inferhub.devart.solutions
• Blog: https://blog.devart.solutions

License

MIT — see LICENSE.