Themis.Inference.Ollama 1.5.1

Themis.Inference.Ollama

Use Themis.Inference.Ollama when an application needs conversational inference or typed System One decisions through an Ollama server, without the ONNX Runtime and tokenizer dependencies used by Themis.Inference.

The package provides two separate integrations: the conversational IInferenceProvider and the typed IDecisionInferenceProvider. Register only the path the application needs.

Conversational inference

AddThemisOllamaChat registers the chat provider and IOllamaModelCatalog:

using Themis.Inference.Ollama.Extensions;

services.AddThemisOllamaChat(options =>
{
    options.Endpoint = new Uri("http://localhost:11434/");
    options.DefaultModel = "qwen2.5:latest";
});

The selected chat model must be available from the configured Ollama server. It can also be supplied on each InferenceRequest.

System One decision inference

Use AddThemisOllamaSystemOneDecisionInference to register Ollama's typed decision endpoint. Register it through AddThemis's service callback so it replaces the default ONNX decision provider while leaving chat registration independent:

services.AddThemis(
    options =>
    {
        options.DefaultModel = "nimble";
        options.RouteUnspecifiedRequestsToDefaultModel = true;
    },
    configureServices: registrations =>
        registrations.AddThemisOllamaSystemOneDecisionInference(options =>
            options.Endpoint = new Uri("http://localhost:11434/")));

An explicit DecisionRequest.Model takes precedence over ThemisOptions.DefaultModel. The endpoint requires Ollama 0.35.0 or later and a local model trained for System One. System One supports choice, score, and noul answers; choice criteria must contain 2-255 options (the model may impose a lower limit), and score criteria must contain 2-10 ordered levels. It does not support tools, streaming, or generation controls. A generic GGUF or chat model is not automatically a System One model; it must be supported by Ollama and trained/packaged for that decision API.

The adapter sends the original Themis state and namespaced extras in a versioned System One state envelope. The extras.rldc namespace includes the central question and attribute-driven prior results; application-defined namespaces are preserved. Task and Language remain routing hints and are not sent to the model. The System One model must be prepared to consume this envelope.

The adapter maps result probabilities to Themis answer-confidence semantics rather than reusing System One's distribution concentration value. System One input/output token usage is retained in DecisionUsage.

Themis.Inference continues to expose the existing AddThemisOllama registration for chat compatibility. Applications that need only the chat integration can reference this package directly and use AddThemisOllamaChat.