Lokad.SafeTensors 0.1.0

Lokad.SafeTensors

Lokad.SafeTensors is a focused C#/.NET implementation of the safetensors file format.

It is designed for model-state persistence scenarios where strict validation, deterministic serialization, and lightweight header inspection matter.

Example

using Lokad.SafeTensors;

var header = SafeTensorSerializer.ReadHeader("weights.safetensors");
foreach (var tensor in header.Tensors)
{
    Console.WriteLine($"{tensor.Name} {tensor.Dtype} [{string.Join(", ", tensor.Shape)}]");
}

var file = SafeTensorSerializer.ReadFile("weights.safetensors");
var weights = file.GetTensor("weights");
var data = weights.GetBytes();

This is the typical consumption flow:

• inspect the header first if you only need metadata and tensor shapes
• load the full file only when you need the payload bytes

When you use ReadFile(path), payload access is backed by a memory-mapped file for high-performance reads without eagerly loading the entire tensor section into managed memory.

Targets

The package currently targets:

• .NET 8
• .NET 10

Scope

Supported:

• one-file .safetensors format
• header-only loading without reading tensor payload bytes
• deterministic writer ordering
• strict validation for offsets, coverage, shape/data size consistency
• metadata key __metadata__ as string-to-string map
• common tensor dtypes used for model states

Not included:

• framework-specific tensor bindings
• compression or encryption
• sub-byte dtype support
• tensor slicing helpers

Main API

• SafeTensorSerializer.Serialize(...)
• SafeTensorSerializer.Deserialize(...)
• SafeTensorSerializer.WriteFile(...)
• SafeTensorSerializer.ReadFile(...)
• SafeTensorSerializer.ReadHeader(...)

ReadHeader(...) is the lightweight path: it validates the file layout and returns metadata plus tensor descriptors without loading tensor payload bytes.

ReadFile(path) is the high-performance file-backed path. Deserialize(bytes) is the in-memory path when the .safetensors payload is already held in memory.

Notes

• serialization is deterministic for tensor and metadata ordering
• validation is strict and fails fast on malformed files
• format issues throw SafeTensorFormatException