Tint 0.2.2

Tint

TextMate syntax colorization for .NET, built for speed, NativeAOT and trimming.

Playground: every bundled grammar and theme, in the browser.

Tint tokenizes and colors source code with the same grammars and themes VS Code uses. Grammars and themes are compiled into your assembly by a source generator and validated at build time; regexes run on GregsEx, a managed Oniguruma-compatible engine, with optional ahead-of-time compiled kernels. There is no native dependency and no JSON parsing at runtime, so it runs anywhere .NET runs, including WebAssembly.

Version 0.1.0. Releases are v*.*.* tags on nuget.org; main publishes previews to Feedz (docs/release.md).

Install

dotnet add package Tint            # runtime, loaders, renderers and the source generator
dotnet add package Tint.Grammars   # optional: the VS Code grammars, already generated
dotnet add package Tint.Themes     # optional: the VS Code color themes, already generated

The Tint runtime also targets .NET Standard 2.0 for .NET Framework hosts. This target supports runtime grammar loading and tokenization but disables runtime regex compilation and modern AOT/trimming annotations. The repository includes a deliberately non-secret development strong-name key, Tint.Development.snk, so signed .NET Framework hosts can reference its assemblies. This key supplies development assembly identity, not production signing or package authenticity. The framework tests run separately on Windows; cross-platform solution tests remain on modern .NET.

Quick start

Import the grammars and themes you need. The source generator turns each file into a class at build time and checks every regex, so a bad pattern is a build error, not a runtime surprise.

<ItemGroup>
  <TintGrammar Include="grammars/csharp.tmLanguage.json" />
  <TintGrammar Include="grammars/JSON.tmLanguage.json" LanguageId="json" FileExtensions=".json;.jsonc" />
  <TintTheme Include="themes/dark_plus.json" />
  
  <AdditionalFiles Include="themes/dark_vs.json" />
</ItemGroup>

Bind a file to a class of your own with an attribute (the path is relative to the project), or let the item alone generate one: source.json becomes JsonGrammar and dark_plus.json becomes DarkPlusTheme, in the project's root namespace.

using Tint;

[TextMateGrammar("grammars/csharp.tmLanguage.json", LanguageId = "csharp", FileExtensions = [".cs"])]
public partial class CSharpGrammar;

var registry = new GrammarRegistry();
registry.Add(CSharpGrammar.Instance, JsonGrammar.Instance);
var darkPlus = Theme.FromSource(DarkPlusTheme.Instance); // resolve once, share everywhere

var highlighter = registry.CreateHighlighter("source.cs", darkPlus);
string html = highlighter.ToHtml(code);        // <pre class="tint" …><code><span class="line">…
Console.Write(highlighter.ToAnsi(code));       // 24-bit terminal colors
highlighter.ToHtml(code.AsSpan(), textWriter); // or stream to a TextWriter / IBufferWriter<char>

A Highlighter owns its tokenizers, so it is not thread-safe: keep one per thread and reuse it.

Patterns are parsed on first use. For NativeAOT or WebAssembly apps with a few hot grammars, opt into ahead-of-time regex kernels with Kernels = true on the attribute, Kernels="true" on the item, or <TintKernels>true</TintKernels> for the project. Items, metadata, diagnostics and kernels are described in docs/generator.md.

Grammar files come from any TextMate source: the VS Code extensions or shiki's textmate-grammars-themes. JSON and plist grammars, VS Code JSON themes (with include) and .tmTheme files are supported.

Bundled grammars and themes

Tint.Grammars and Tint.Themes ship the VS Code grammars (84 files, from 51 extensions) and its 19 color themes already generated, for quick starts, demos and the playground. No items or generator setup needed. The playground (samples/Tint.Playground) registers all of them in a Blazor WebAssembly app compiled ahead of time.

dotnet add package Tint.Grammars
dotnet add package Tint.Themes

using Tint;
using Tint.Grammars;
using Tint.Themes;

var registry = new GrammarRegistry();
registry.Add(TintGrammars.CSharp, TintGrammars.Json); // or registry.Add([.. TintGrammars.All])

var tokenizer = registry.CreateTokenizer("source.cs", Theme.FromSource(TintThemes.DarkPlus));

IGrammarSource? markdown = TintGrammars.Find("text.html.markdown"); // by scope name: TintGrammars.Markdown
IThemeSource? monokai = TintThemes.Find("Monokai");                // by theme name or property name

Each grammar and theme is a class (CSharpGrammar, DarkPlusTheme, ...) with a shared Instance; TintGrammars and TintThemes expose one property per class, named after it without the suffix (TintGrammars.CSharp, TintGrammars.Markdown, TintGrammars.TypeScriptReact, TintThemes.DarkPlus, TintThemes.LightVisualStudio), All and Find. Grammar classes are named after their language (ShellScriptGrammar for shellscript, PugGrammar for jade); grammars without one, included by others, after their scope (HtmlBasicGrammar, Yaml12Grammar, JsDocTypeScriptInjectionGrammar). Patterns are parsed on first use.

Only the classes you reference stay in a trimmed or NativeAOT build: an app that uses TintGrammars.CSharp, TintGrammars.Json and TintThemes.DarkPlus trims Tint.Grammars.dll from 5.2 MB to 146 KB. Reading All or calling Find keeps every class, as it must. A grammar that includes another (Markdown fenced blocks, HTML with CSS and JavaScript) needs that one registered too; registry.GetMissingGrammars(scopeName) lists what is missing.

Generating from the VS Code sources

To compile only the grammars and themes you use, with your own class names and optional kernels, reference the file-only Tint.Grammars.Sources and Tint.Themes.Sources packages (development dependencies, nothing ships with your app) and include the files from their folders:

<PackageReference Include="Tint" Version="..." />
<PackageReference Include="Tint.Grammars.Sources" Version="..." PrivateAssets="all" />
<PackageReference Include="Tint.Themes.Sources" Version="..." PrivateAssets="all" />
<TintGrammar Include="$(TintGrammarsSourcesPath)csharp/syntaxes/csharp.tmLanguage.json" Kernels="true" />
<TintTheme Include="$(TintThemesSourcesPath)theme-defaults/dark_plus.json" />
<AdditionalFiles Include="$(TintThemesSourcesPath)theme-defaults/dark_vs.json" />

@(TintGrammarsSourcesAll) and @(TintThemesSourcesAll) list every file with the bundles' metadata; see docs/generator.md.

Tokens

For editors and custom renderers, work with tokens directly. Line state carries over between lines, so a document can be re-tokenized incrementally from the first changed line.

var tokenizer = registry.CreateTokenizer("source.cs", DarkPlusTheme.Instance);
var state = StateStack.Initial;

foreach (var line in document.Lines)
{
    foreach (var token in tokenizer.TokenizeLine(line, state, out state))
    {
        // token.Start, token.Length, token.Scopes, token.Foreground, token.FontStyle
    }
}

TokenizeLineEncoded returns (start, metadata) pairs in the VS Code binary layout for hosts that already consume Monaco-style tokens. highlighter.Tokenize(code) returns one HighlightedToken[] per line with the colors resolved (Start, Length, Color, BackgroundColor, FontStyle, Scopes).

A Tokenizer is not thread-safe and builds its regex scanners as it reaches rules, so create one per thread or per open document and keep it alive; the compiled grammar behind it is shared.

HTML

ToHtml writes markup in the shape shiki made common, so stylesheets and tools written for shiki output apply unchanged: the theme's colors on the pre, one <span class="line"> per line, one <span style="…"> per run of tokens with the same style holding only what differs from the theme's defaults, text HTML-escaped. A rendered sample and the exact conventions are in docs/html-output.md. HtmlRendererOptions sets the pre class, data-line line numbers, and a CSS-class mode with a matching stylesheet:

var highlighter = registry.CreateHighlighter("source.cs", darkPlus, new HighlighterOptions
{
    Html = new HtmlRendererOptions { StyleMode = HtmlStyleMode.CssClasses }, // class="tint-fg-4 tint-b"
});
string css = highlighter.GetStyleSheet();

With a light and a dark theme, the light colors are written inline and the dark ones as CSS variables (--tint-dark, --tint-dark-bg, --tint-dark-font-style…), so one CSS rule switches themes:

var highlighter = registry.CreateHighlighter("source.cs",
    light: Theme.FromSource(LightPlusTheme.Instance), dark: darkPlus);

string html = highlighter.ToHtml(code);

@media (prefers-color-scheme: dark) {
  .tint, .tint span {
    color: var(--tint-dark) !important;
    background-color: var(--tint-dark-bg) !important;
    font-style: var(--tint-dark-font-style) !important;
    font-weight: var(--tint-dark-font-weight) !important;
    text-decoration: var(--tint-dark-text-decoration) !important;
  }
}

More named themes work the same way (CreateHighlighter(scope, defaultTheme, [new("dim", dim)]) writes --tint-dim). HtmlRenderer and AnsiRenderer in Tint.Rendering can also be used directly with tokenizers of your own.

Loading grammar files at runtime

When grammars are only known at runtime, load them instead of generating them. Same tokens, regexes compiled on first use.

var grammar = GrammarLoader.Load("my-language.tmLanguage.json");
registry.Add(grammar);

Performance

Tokenizing TextMateSharp's bigfile.cs (133,439 lines, 5.8 MB) with the VS Code C# grammar and Dark+, once warm, through each library's encoded-token path:

Measured on an Apple M4 Pro with .NET 10.0.11, GregsEx 0.5.2 and BenchmarkDotNet in-process (2 warmups, 3 iterations). The bundles run interpreted; ahead-of-time kernels are an opt-in for grammars you import (Kernels="true"): with GregsEx 0.5.2 they cover 327 of the C# grammar's 338 patterns (the subexpression-call declarations included) and tokenize this file in 0.75 of the interpreted time, while costing about 150 ms of JIT on first use in a fresh process. The full set, with methodology, is in benchmarks/README.md.

Build

dotnet build
dotnet test
dotnet run -c Release --project benchmarks/Tint.Benchmarks

Grammars and themes under grammars/ and themes/ are synced from a pinned VS Code commit, which also regenerates THIRD-PARTY-NOTICES.txt:

dotnet run --project tools/Tint.Sync -- sync --commit 3ac0f056

Design notes, the plan and research are indexed in docs/README.md.

License

MIT. Bundled grammars and themes keep their original licenses, listed in THIRD-PARTY-NOTICES.txt.