Warpstone.Sources 3.0.0-preview04

This is a prerelease version of Warpstone.Sources.
dotnet add package Warpstone.Sources --version 3.0.0-preview04
                    
NuGet\Install-Package Warpstone.Sources -Version 3.0.0-preview04
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Warpstone.Sources" Version="3.0.0-preview04">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="Warpstone.Sources" Version="3.0.0-preview04" />
                    
Directory.Packages.props
<PackageReference Include="Warpstone.Sources">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add Warpstone.Sources --version 3.0.0-preview04
                    
#r "nuget: Warpstone.Sources, 3.0.0-preview04"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package Warpstone.Sources@3.0.0-preview04
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=Warpstone.Sources&version=3.0.0-preview04&prerelease
                    
Install as a Cake Addin
#tool nuget:?package=Warpstone.Sources&version=3.0.0-preview04&prerelease
                    
Install as a Cake Tool

Buy Me A Coffee

Package Stable Preview Downloads
Warpstone NuGet NuGet NuGet
Warpstone.Sources NuGet NuGet NuGet

<img src="https://raw.githubusercontent.com/CptWesley/Warpstone/main/logo_1024x1024.png" width="178" height="178">

Warpstone

Parser combinator forged deep within Ikit's forges in the Under-City from a shard of Morrslieb itself.

What is it?

Warpstone is a parser combinator library written in C# targetting .NET Standard 2.0, meaning that it can be used by any application running either .NET Framework >= 4.6.1, .NET Core >= 2.0 or Mono >= 5.4. The main focus of the library is to provide a powerful lightweight parser combinator framework for developers to create their own parsers with.

Why did you make it?

After following a course on compiler construction I was intrigued and wanted a way of parsing languages within code without requiring external tools to generate a parser for me first. During my quest I stumbled upon the concept of parser combinators and started looking for existing libraries for .NET. I soon stumbled upon Pidgin. After toying around with the library for a bit I felt dissatisfied with some of its syntax and I still felt like I didn't fully understand the concepts it was using behind the scenes. This led me to create my own parser combinator library, with syntax inspired by Pidgin.

Why the name? (Or: What are all those strange words in the description?!?!?)

Coming up with names is difficult, I wanted to get the project started and at the time I was consumed by the Warhammer Fantasy universe.

Where do the ideas come from?

Warpstone's implementation draws on the following work:

How does it compare?

✅ supported, ðŸŸĄ partially supported, ❌ not supported.

Feature Warpstone V1 Warpstone V2 Warpstone V3 Pidgin Parlot ParsecSharp Farkle
Left recursion ❌ ❌ ✅ ❌ ❌ ❌ ✅
Packrat memoization ❌ ✅ ✅ ❌ ❌ ❌ ❌<sup>1</sup>
Backtracking choice (no Try needed) ✅ ✅ ✅ ðŸŸĄ<sup>2</sup> ✅ ✅ ❌<sup>1</sup>
Lookahead (positive/negative) ✅ ✅ ✅ ✅ ✅ ✅ ❌<sup>1</sup>
Deeply nested input ❌ ✅ ✅ ❌ ❌ ✅ ✅
Cancellation ❌ ✅ ✅ ❌ ✅ ❌ ❌
Step-wise execution ❌ ✅ ✅ ❌ ❌ ❌ ❌
Regex parsers ✅ ✅ ✅ ❌ ❌ ❌ ✅
Operator precedence helpers ✅ ❌ ❌<sup>3</sup> ✅ ✅ ðŸŸĄ<sup>4</sup> ✅
Monadic bind (context-sensitive parsing) ❌ ✅ ðŸŸĄ<sup>5</sup> ✅ ðŸŸĄ<sup>6</sup> ✅ ❌
Non-char token input ❌ ❌ ❌ ✅ ❌ ✅ ðŸŸĄ<sup>7</sup>
Streaming input (TextReader/Stream) ❌ ❌ ✅ ✅ ✅ ✅ ✅
Compiled / source-generated parsers ❌ ❌ ❌ ❌ ✅ ❌ ✅
Source-only package (no assembly dependency) ❌ ❌ ✅ ❌ ❌ ❌ ❌
  1. Farkle is an LALR(1) parser generator, not a PEG/combinator library. It needs no memoization or backtracking and reports grammar conflicts when the grammar is built.
  2. A choice only backtracks over consumed input when the branch is wrapped in Try.
  3. Not needed: write expressions as left-recursive rules.
  4. ChainLeft/ChainRight combinators, no precedence table.
  5. Via Bind. Left recursion through the selected parser is not grown and fails with an InfiniteRecursionError; a selected parser that is not part of the grammar is prepared for parsing each time a new instance is returned.
  6. Via Switch.
  7. Custom tokenizers, input is still text.

Third-party columns reflect each library's documentation and package API at the time of writing (Pidgin 3.5, Parlot 2.3, ParsecSharp 5.2, Farkle 7).

Performance

Every library parses the same MiniJava subset into the same AST (MiniJavaBenchmark). Inputs: small (619 chars), medium (7,286 chars) and large (73,241 chars). Lower is better.

Mean CPU time per parse:

Library Small Medium Large
Parlot (source-generated) 8.1 Ξs 92.7 Ξs 1.00 ms
Parlot (runtime) 10.4 Ξs 124.3 Ξs 1.37 ms
Farkle 17.1 Ξs 208.1 Ξs 2.12 ms
Warpstone V3 24.7 Ξs 297.1 Ξs 2.86 ms
Warpstone V3 (left-recursive)<sup>7</sup> 40.6 Ξs 524.0 Ξs 5.29 ms
Pidgin 101.2 Ξs 1.27 ms 13.52 ms
Warpstone V1 487.5 Ξs 6.21 ms 60.29 ms
ParsecSharp 549.1 Ξs 6.84 ms 67.45 ms
Warpstone V2 3.39 ms 79.51 ms 805.98 ms

Memory allocated per parse:

Library Small Medium Large
Farkle 6.5 KB 65.0 KB 650 KB
Parlot (source-generated) 6.5 KB 67.5 KB 673 KB
Parlot (runtime) 6.6 KB 67.5 KB 673 KB
Warpstone V3 14.7 KB 93.9 KB 866 KB
Warpstone V3 (left-recursive)<sup>7</sup> 22.1 KB 179.2 KB 1,696 KB
Pidgin 21.8 KB 252.0 KB 2,523 KB
Warpstone V1 1,578 KB 19,116 KB 191,131 KB
ParsecSharp 1,956 KB 24,179 KB 242,814 KB
Warpstone V2 9,069 KB 109,981 KB 1,104,776 KB
  1. Binary operators written as left-recursive rules instead of repetitions.

Measured with BenchmarkDotNet 0.16.0-preview.2 (InProcessEmitToolchain) on .NET 11.0, AMD Ryzen 9 7950X3D, Windows 11. Reproduce from src/ with dotnet run -c Release --project Warpstone.Benchmarks -- --filter "*MiniJavaBenchmark*" --inProcess.

Usage

Install the package. V3 is in preview, so include prereleases:

dotnet add package Warpstone --prerelease

Or install Warpstone.Sources instead, which compiles the library's source code into your project, so your build output has no Warpstone.dll dependency.

Example: a calculator

The program below parses and evaluates arithmetic such as (1 + 2) * 3. It's a complete console app: paste it into Program.cs and run it.

using System;
using Warpstone;
using Warpstone.Errors;
using static Warpstone.Parsers;

// Parse() returns the value and throws if the input is invalid.
Console.WriteLine(Calculator.Expression.Parse("1 + 2 * 3"));   // 7
Console.WriteLine(Calculator.Expression.Parse("(1 + 2) * 3")); // 9
Console.WriteLine(Calculator.Expression.Parse("10 - 4 - 3"));  // 3: '-' groups to the left

// TryParse() doesn't throw. The result holds either a value or the errors.
var result = Calculator.Expression.TryParse("1 + * 2");
if (result.Success)
{
    Console.WriteLine(result.Value);
}
else
{
    foreach (var error in result.Errors)
    {
        Console.WriteLine($"Error at {error.InputStartPosition}");
        if (error is UnexpectedTokenError unexpected)
        {
            Console.WriteLine($"Expected {string.Join(" or ", unexpected.Expected)} but found {unexpected.Found}");
        }
    }
}

public static class Calculator
{
    // A number is one or more digits. Regex() matches the text, Transform() converts it to an int.
    // WithName() sets the name used in error messages instead of the regex pattern.
    private static readonly IParser<int> Number
        = Regex("[0-9]+").Transform(int.Parse).WithName("number");

    // An atom is a number or a sum between parentheses.
    // Sum is declared further down, so Lazy() postpones reading the field until parsing starts.
    // The '!' tells the compiler the field will be set by then.
    // Then() keeps only the right-hand result, ThenSkip() keeps only the left-hand result.
    private static readonly IParser<int> Atom
        = Or(
            Number,
            Char('(').Then(Lazy(() => Sum!)).ThenSkip(Char(')')));

    // A term is atoms joined by '*' or '/'.
    // The rule starts with itself (left recursion). Warpstone detects and supports this, so
    // operators group to the left: 8 / 4 / 2 means (8 / 4) / 2.
    // ThenAdd() keeps both results as a tuple, which Transform() unpacks into two arguments.
    // Or() tries each alternative in order and returns the first that succeeds.
    private static readonly IParser<int> Term
        = Or(
            Lazy(() => Term!).ThenSkip(Operator("*")).ThenAdd(Atom).Transform((left, right) => left * right),
            Lazy(() => Term!).ThenSkip(Operator("/")).ThenAdd(Atom).Transform((left, right) => left / right),
            Atom);

    // A sum is terms joined by '+' or '-'. Because a sum is built from terms,
    // '*' and '/' bind tighter than '+' and '-'.
    private static readonly IParser<int> Sum
        = Or(
            Lazy(() => Sum!).ThenSkip(Operator("+")).ThenAdd(Term).Transform((left, right) => left + right),
            Lazy(() => Sum!).ThenSkip(Operator("-")).ThenAdd(Term).Transform((left, right) => left - right),
            Term);

    // The entry point. ThenEnd() fails if any input is left over, so "1 + 2 abc" is rejected.
    public static readonly IParser<int> Expression
        = Sum.ThenEnd();

    // An operator symbol with optional whitespace on both sides.
    private static IParser<string> Operator(string symbol)
        => OptionalWhitespaces.Then(String(symbol)).ThenSkip(OptionalWhitespaces);
}

Output:

7
9
3
Error at Index 4 Line 1 Column 5
Expected number or '(' but found '*'
There are no supported framework assets in this package.

Learn more about Target Frameworks and .NET Standard.

  • .NETStandard 2.0

    • No dependencies.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
3.0.0-preview04 0 11/20/2026
3.0.0-preview03 441 6/11/2025
3.0.0-preview02 354 6/11/2025

### v3.0.0
       - Significantly improved performance.
       - Reworked API interface.
       ### v2.0.0
       - Reworked library from scratch.
       - Changed to Packrat parsing.
       - Added support for left-recursion.