TitleFactory 26.6.1856

TitleFactory — pure C# chat title generation

A dependency-free^* .NET 10 implementation of a compact 350M-parameter model purpose-built for generating chat conversation titles (upstream model on HuggingFace). Everything — GGUF parsing, K-quant dequantization, byte-level BPE tokenization, and the full LFM2 forward pass — is implemented in managed C# with SIMD acceleration (System.Numerics.Tensors.TensorPrimitives, AVX2/AVX-512 capable).

^* the only package reference is System.Numerics.Tensors.

using TitleFactory;

var model = TitleModel.LoadEmbedded(); // Q4_K_M weights ship inside the assembly

string title = model.GenerateTitle("Hey, my wifi keeps dropping every few minutes, how can I fix it?");
// -> "Fixing Wifi Dropdown Issues"

// or with full conversations:
title = model.GenerateTitle([
    ChatMessage.User("I need a recipe for a quick vegetarian dinner tonight"),
    ChatMessage.Assistant("How about a 20-minute chickpea curry? ..."),
    ChatMessage.User("Sounds great, can I substitute the coconut milk?"),
]);
// -> "Coconut Milk Substitution"

On a 4-core container a title takes roughly 1–2.5 seconds (greedy decoding); model load is about 1.5 s. Per the model card, do not include a system prompt — the model is not trained with one.

What's inside

The model is a fine-tune of LiquidAI LFM2.5-350M, using the hybrid LFM2 architecture: 16 blocks, of which 10 are double-gated short-range convolution blocks (out_proj(C ⊙ causal_conv3(B ⊙ x))) and 6 are grouped-query attention blocks (16 heads / 8 KV heads, per-head QK RMS-norm, NeoX RoPE θ=10⁶), each followed by a SwiGLU MLP (dim 4608). Hidden size 1024, vocab 65536, tied embeddings.

The recommended Q4_K_M quantization (229 MB) is embedded in the library assembly as split resources (src/TitleFactory/Resources/), so TitleModel.LoadEmbedded() works with no external files. Any other quantization of the model can be loaded from disk with TitleModel.Load(path) (F32/F16/BF16/Q8_0/Q4_K_M/Q6_K files are supported).

Validation

The implementation is validated layer by layer against the official transformers Lfm2 implementation:

1. tools/extract_reference.py dequantizes the GGUF weights with gguf-py (the llama.cpp reference dequantization), loads them into Lfm2ForCausalLM, and captures ~93 intermediate activations per quantization (embeddings, every block's operator norm / conv / attention / FFN outputs, fine-grained conv & attention internals, final norm, logits) plus greedy-decoding references and tokenizer test vectors. These are committed under tests/TitleFactory.Tests/TestData/.
2. The C# test-suite re-runs the same prompt through the managed implementation and compares every captured tensor (relative tolerance 2e-3), for both the embedded Q4_K_M weights and (optionally) the F16 weights.
3. End-to-end tests check that greedy generation reproduces the reference titles token-for-token, and that incremental (KV/conv-cached) decoding matches batch prefill exactly.

dotnet test                          # full suite using the embedded Q4_K_M model
TITLE_FACTORY_F16_GGUF=path/to/f16.gguf dotnet test   # + high-precision F16 validation
TITLE_FACTORY_NO_DOWNLOAD=1 dotnet test               # never download the F16 file

To regenerate the reference data:

pip install gguf numpy torch transformers
python3 tools/extract_reference.py \
    --f16 title-model-f16.gguf --q4km title-model-q4km.gguf \
    --tokenizer tokenizer.json --out tests/TitleFactory.Tests/TestData

Demo

dotnet run --project examples/TitleFactory.Demo -c Release -- "how do I center a div?"

Licenses

The code in this repository is MIT-licensed. The model weights (embedded in src/TitleFactory/Resources/ and downloaded for tests) are published by their authors under GPL-3.0 (see the model link above); review that license before redistributing the built assembly with the embedded weights.