LLMFromScratch.SDK.ThirdPartyTraining 0.1.4

LLMFromScratch third-party training extension

LLMFromScratch.SDK.ThirdPartyTraining is the opt-in extension for training Hugging Face Qwen, Llama, and compatible decoder models through Python tooling. It is deliberately separate from the original in-process GPT SDK.

Install it alongside the base SDK:

dotnet add package LLMFromScratch.SDK --version 0.3.2
dotnet add package LLMFromScratch.SDK.ThirdPartyTraining --version 0.1.4

The extension provides ThirdPartyModelPipeline for Unsloth or PEFT LoRA/QLoRA training, llama.cpp GGUF conversion, and local Ollama import.

Complete pipeline example

using System.Collections.Generic;
using LLMFromScratch.SDK.ThirdPartyTraining;

var pipeline = new ThirdPartyModelPipeline();
var result = await pipeline.RunAsync(
    new ThirdPartyModelPipelineRequest
    {
        Training = new ThirdPartyTrainingOptions
        {
            ModelId = "Qwen/Qwen2.5-0.5B-Instruct",
            ModelFamily = ThirdPartyModelFamily.Qwen,
            Backend = ThirdPartyTrainingBackend.Unsloth,
            DatasetPath = "Data/train.jsonl",
            OutputDirectory = "outputs/qwen-support",
            Epochs = 1,
            BatchSize = 1,
            GradientAccumulationSteps = 8,
            MaxSequenceLength = 2048,
            LoadIn4Bit = true,
            MergeAdapter = true
        },
        GgufExport = new ThirdPartyGgufExportOptions
        {
            // Filled with the merged training output by the pipeline.
            OutputPath = "outputs/qwen-support/qwen-support-q4_k_m.gguf",
            LlamaCppDirectory = @"C:\src\llama.cpp",
            Quantization = "Q4_K_M"
        },
        OllamaImport = new OllamaImportOptions
        {
            // GgufPath is filled by the GGUF stage.
            ModelName = "qwen-support",
            ModelfilePath = "outputs/qwen-support/Modelfile.qwen-support",
            SystemPrompt = "You are a concise, helpful support assistant.",
            Parameters = new Dictionary<string, string>
            {
                ["temperature"] = "0.2",
                ["num_ctx"] = "4096"
            }
        }
    },
    new Progress<string>(Console.WriteLine));

Console.WriteLine($"GGUF: {result.GgufExport?.OutputPath}");
Console.WriteLine($"Ollama model: {result.OllamaImport?.ModelName}");
Console.WriteLine($"Modelfile: {result.OllamaImport?.ModelfilePath}");

OllamaImportOptions writes the Modelfile and then runs ollama create <model-name> --file <modelfile-path>. The generated file uses the absolute GGUF path, supplied PARAMETER values, and optional SYSTEM or TEMPLATE directives. Leave Template unset unless you intentionally need to override the model's chat template.

For a manual import, the generated Modelfile has this form:

FROM C:/absolute/path/to/outputs/qwen-support/qwen-support-q4_k_m.gguf
PARAMETER temperature 0.2
PARAMETER num_ctx 4096
SYSTEM """You are a concise, helpful support assistant."""

ollama create qwen-support:latest --file outputs/qwen-support/Modelfile.qwen-support

For Python, llama.cpp, Ollama, dataset, standalone import, and manual Modelfile guidance, see the repository documentation at docs/ThirdPartyModels.md.

MIT. See the repository license.