LLMFromScratch.SDK 0.3.2

LLMFromScratch SDK

LLMFromScratch.SDK is the base .NET 9/10 SDK for this repository's original in-process GPT model. It creates, trains, checkpoints, tool-enables, and exports that model as lossless F32 GGUF. It has no Qwen, Llama, Hugging Face, Python, Unsloth, PEFT, llama.cpp, or Ollama training dependency.

Install

dotnet add package LLMFromScratch.SDK --version 0.3.2

Add exactly one native TorchSharp runtime package for the target host:

Do not mix CPU and CUDA runtime payloads in the same application. On Apple Silicon, target osx-arm64; MPS/Metal is supplied by macOS.

Train the native GPT model

Create Data/train.txt, optionally Data/validation.txt, and matching GPT-2 Models/vocab.json and Models/merges.txt files. Then configure the client:

using LLMFromScratch.Core.Configurations;
using LLMFromScratch.SDK;
using LLMFromScratch.Training.Export;

var options = new LlmFromScratchOptions
{
    Model = new GPTModelConfiguration
    {
        VocabularySize = 50_257,
        ContextLength = 128,
        EmbeddingDimension = 256,
        NumberOfHeads = 8,
        NumberOfLayers = 4,
        Dropout = 0.1
    },
    Training = new TrainingOptions
    {
        Epochs = 1,
        BatchSize = 1,
        LearningRate = 3e-4,
        MinimumLearningRate = 3e-4,
        WeightDecay = 0.01,
        GradientClip = 1.0,
        Optimizer = OptimizerType.AdamW,
        Scheduler = SchedulerType.Constant
    },
    ComputeDevice = new ComputeDeviceOptions
    {
        Preference = ComputeDevicePreference.Auto
    }
};

using var client = new LlmFromScratchClient(options);
client.SetDatasetPaths("Data/train.txt", "Data/validation.txt");

var training = await client.TrainAsync();
var gguf = client.ExportGguf(new GgufExportOptions
{
    VocabularyPath = "Models/vocab.json",
    MergeFilePath = "Models/merges.txt",
    OutputPath = "exports/my-gpt-f32.gguf",
    ModelName = "My GPT model"
});

Console.WriteLine($"Loss: {training.Loss:F4}");
Console.WriteLine($"GGUF: {gguf.OutputPath}");
Console.WriteLine($"Modelfile: {gguf.ModelfilePath}");

TrainAsync writes a checkpoint after each epoch. ExportGguf refuses to overwrite an existing output file. Use it only for the LLMFromScratch GPT-2-compatible model, whose tensor layout and tokenizer are not compatible with Qwen, Llama, or other Hugging Face models.

ExportGguf also writes a companion Ollama Modelfile next to the GGUF file (<output>.Modelfile by default), built from the checkpoint's own llmfromscratch.* control-token metadata so ollama create picks up a matching TEMPLATE and stop sequences automatically. Set GgufExportOptions.WriteModelfile = false to skip it, or GgufExportOptions.ModelfilePath to choose the path; the path actually written is reported on GgufExportResult.ModelfilePath. This only fixes the prompt format Ollama runs the model with — it does not change what the underlying checkpoint has learned, so a model that needs more or better training data will still answer incorrectly inside a correctly-templated prompt.

Optional third-party training extension

Install LLMFromScratch.SDK.ThirdPartyTraining only when you need Qwen/Llama LoRA or QLoRA training through Unsloth or PEFT, llama.cpp GGUF conversion, or Ollama import. That opt-in package has its own pipeline API and Python tooling; it does not change the base SDK's model or device lifecycle. See the repository docs/ThirdPartyModels.md for its prerequisites and examples.

Device and lifecycle notes

• ComputeDevicePreference.Auto selects CUDA, then Apple Silicon MPS, then CPU.
• ComputeDevicePreference.Cuda fails early if CUDA is unavailable.
• Dispose LlmFromScratchClient to release model and native tensor resources.

MIT. See the repository license.