BorrowedTorch.Persistency
1.1.0
Prefix Reserved
dotnet add package BorrowedTorch.Persistency --version 1.1.0
NuGet\Install-Package BorrowedTorch.Persistency -Version 1.1.0
<PackageReference Include="BorrowedTorch.Persistency" Version="1.1.0" />
<PackageVersion Include="BorrowedTorch.Persistency" Version="1.1.0" />
<PackageReference Include="BorrowedTorch.Persistency" />
paket add BorrowedTorch.Persistency --version 1.1.0
#r "nuget: BorrowedTorch.Persistency, 1.1.0"
#:package BorrowedTorch.Persistency@1.1.0
#addin nuget:?package=BorrowedTorch.Persistency&version=1.1.0
#tool nuget:?package=BorrowedTorch.Persistency&version=1.1.0
BorrowedTorch.Persistency
Persistency for the Borrowed Torch engine: documents of a format-neutral data tree, written as JSON today, with versioned migrations, an object mapper that can be taught any type, and project folders that keep their files together.
- Documents that say what they are (
kind) and which layout they were written in (version), saved atomically - A data tree of objects, arrays and values that compares by content, with numbers kept exactly as the type they came from wrote them
- Diffs and patches between two trees - entities matched by their ids, not their places - applied either way, for an undo that keeps changes rather than copies
- A JSON codec that writes the same bytes for the same tree, readable in a diff; a binary codec can sit beside it later
- Migrations that take older documents to the current layout step by step, and refuse newer ones
- An object mapper for fields, properties, records and collections, taught other types by converters
- Unknown data kept as read and written back, so opening a file with less than wrote it loses nothing
- Project folders whose files are referred to by relative paths, and brought in by copying
The package targets
net10.0and depends on nothing but the .NET libraries.
Installation
dotnet add package BorrowedTorch.Persistency
Quick start
using BorrowedTorch.Persistency;
using BorrowedTorch.Persistency.Data;
using BorrowedTorch.Persistency.Serialization;
public struct Light
{
public float Intensity;
public string Name;
public static readonly Light Default = new() { Intensity = 5, Name = "light" };
}
// Objects to a tree: a member the data lacks keeps the default it starts from.
var serializer = new DataSerializer(new DataSerializerOptions
{
DefaultFactory = type => type == typeof(Light) ? Light.Default : null
});
var data = new DataObject().Add("light", serializer.Write(new Light { Intensity = 0.1f, Name = "sun" }));
// A document, saved atomically.
DocumentFile.Save("Scenes/Main.btscene", new Document("game.scene", 1, data));
// Read back, upgraded from older layouts on the way.
var migrations = new MigrationSet("game.scene", 1);
var document = migrations.Upgrade(DocumentFile.Load("Scenes/Main.btscene"));
var light = serializer.Read<Light>(document.Data["light"]);
The file it writes:
{
"kind": "game.scene",
"version": 1,
"data": {
"light": {
"Intensity": 0.1,
"Name": "sun"
}
}
}
Concepts
Documents
A Document is a kind, a version and its data (a DataObject). DocumentFile saves and loads them:
| Member | Purpose |
|---|---|
Save(path, document, codec) |
Writes the whole document to a temporary file beside the target, flushes it to disk, then puts it in the target's place. A failure midway leaves the old file untouched. Creates the folder. |
Load(path, codecs), Read(stream, codecs) |
Reads a document, choosing the codec by the bytes it begins with. Errors name the file. |
Write(stream, document, codec), ToBytes(document, codec) |
The document's bytes; the same for the same document. |
The data tree
DataNode is a DataObject (named members in the order they were added), a DataArray, or a DataValue (null, boolean, number or string). Trees compare by content (Equals) and copy deeply (Clone), which is what an "unsaved changes" check needs.
A number keeps its text: a float 0.1 is written 0.1 and reads back as exactly that float, and a number read from a file keeps the text it had there. NaN and the infinities are kept by name, written as strings in JSON.
DataObject has Add, Insert (at a place), Set (replaces in place), Remove, Rename (keeps the place), IndexOf and typed getters (GetObject, GetString, TryGetArray, ...) whose errors say what was found instead.
Diffs and patches
DataDiff.Between(before, after, options) gives the DataPatch that makes one tree the other: objects member by member (one added goes back to its place, as member order is part of an object), arrays element by element, values where they differ. An array named in DataDiffOptions.ArrayKeys is matched by one member of its elements - options.ArrayKeys["entities"] = "id" - so an element added, removed or changed among thousands is that one change; others are matched by place, and replaced whole where their lengths differ. Members, or keyed elements, in another order replace their object or array whole.
var options = new DataDiffOptions();
options.ArrayKeys["entities"] = "id";
var patch = DataDiff.Between(before, after, options); // what an edit changed
var undone = patch.Inverse().ApplyTo(after); // equal to before
patch.Apply(tree); // the tree itself changed, in place
| Member | Purpose |
|---|---|
Changes |
Each DataChange: its Path, its Kind (Set, Add, Remove), the node Before and After, the Index it is added at or removed from. |
Inverse() |
The patch that undoes it. |
Apply(tree, strict), ApplyTo(tree, strict) |
The changes done in place - the root returned, another node only where the root itself was replaced - or to a copy. Strict, each change must find what it was made from, or a PersistencyException names the path; not strict, it does what it can. |
Size |
About how many bytes it holds, for a history kept within a budget. |
A DataPath is written as a reader would point at a node - entities[id="3f2a…"].components.NameComponent.Name, Nodes[3] - and DataPath.Parse reads it back.
JSON
JsonDataCodec writes UTF-8 without a byte order mark, indented by two spaces, with LF line ends and a final newline; a short array of plain values (a vector, a colour) goes on one line, [1, 0.5, 0]; non-ASCII text is written as it is. Reading accepts comments, trailing commas and a byte order mark, so hand-edited files read, and refuses a member named twice.
A format of your own implements IDataCodec (CanRead, Read, Write) and is passed to Save and Load.
Migrations
MigrationSet(kind, currentVersion) holds one step per version (Add(fromVersion, data => ...)), each working on the data tree, so it reads layouts whose types are gone. Upgrade(document) copies the data and takes it through every step to the current version; a document already current is returned as it is. A newer version, a missing step or another kind is refused with a DocumentVersionException or PersistencyException. UpgradeData(data, version) does the same in place for data versioned on its own inside a document.
Mapping objects
DataSerializer writes objects to trees and reads them back. Without a converter:
| Type | Written as |
|---|---|
bool, numbers, char, string, Guid, PersistentId |
a value |
| an enum | its name (flags as "Move, Scale"); a number reads too |
arrays, lists, ImmutableArray<T>, other sequences |
an array; a default ImmutableArray is null, an empty one [] |
a dictionary with string keys |
an object, keys in ordinal order |
a DataNode |
itself |
| any other class or struct | an object of its public fields, then its public properties that can be set (set or init) or are passed to its constructor, in declaration order |
Internal members and computed properties are left out, as is anything marked [DataIgnore]; [DataName("old")] keeps reading data written under an old name. A class without a parameterless constructor is made through the public constructor whose parameters all name members, as positional records are.
Reading starts from DataSerializerOptions.DefaultFactory, or from the value passed to ReadInto, and puts in only the members the data has: a member added since the data was written keeps its default, and a member the type has lost is passed over.
What is worked out for a type - its members, the converter it is matched with - is worked out once and kept. A type of an assembly that can be unloaded (one loaded into a collectible AssemblyLoadContext, such as a game's code reloaded) is kept no longer than the type itself, so writing and reading it does not keep its assembly loaded.
Converters (IDataConverter, or DataConverter<T> for one type) are asked first, in order: for abstract types, which the mapper does not write type names for, for types written more simply than their members (a vector as [x, y, z]), and for references written as ids.
DataContext carries what one operation needs: services for converters (Set, Get), work deferred until everything is read (Defer, RunDeferred) such as resolving references, and messages (Report).
Errors say where they happened: Keys[2].Time: Expected a double, found the string "soon".
Ids and unknown data
PersistentId is an id that survives saving and loading, written as 32 hex digits and ordered by value. UnknownData(key, data) holds data read under a key nothing knows, to be written back as it was read.
Project folders
ProjectFolder(root) holds a project's documents and files, which refer to each other by paths relative to the root, with forward slashes:
| Member | Purpose |
|---|---|
Resolve(relative) |
The absolute path; refuses one that leads out of the project. |
MakeRelative(path), Contains(path) |
A path inside the project, relative to it. |
Import(source, subfolder) |
Brings a file in: a file already inside is left where it is; one from outside is copied into the subfolder under its own name, or name (2) where another file has it, unless that file has the same bytes. |
Enumerate(subfolder, extension) |
The files with the extension in a subfolder, in order. |
The layout is yours: which subfolders hold what, and which extensions.
License
MIT
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- No dependencies.
NuGet packages (1)
Showing the top 1 NuGet packages that depend on BorrowedTorch.Persistency:
| Package | Downloads |
|---|---|
|
BorrowedTorch.Engine
Kindling, Borrowed Torch's game engine: what a game's own code builds against - its features registering components, systems, keys and UI handlers - for the Kindling editor and player to run |
GitHub repositories
This package is not used by any popular GitHub repositories.