omy.Utils.Parser.Generators 2.0.0-rc.1

Utils.Parser.Generators

A Roslyn source generator that reads ANTLR4 .g4 grammar files at build time and emits C# code that constructs the Utils.Parser model directly — no runtime .g4 parsing, no reflection, no extra dependencies in your output.

Install

dotnet add package omy.Utils.Parser.Generators --version 2.0.0-rc.1

You also need the runtime library:

dotnet add package omy.Utils.Parser --version 2.0.0-rc.1

Supported frameworks

• netstandard2.0 (Roslyn analyzer / source generator)

---

How it works

1. You declare a .g4 grammar file as an AdditionalFiles item in your .csproj.
2. At build time the generator parses the file with an internal G4 tokenizer and parser.
3. It emits a partial static C# facade whose core members include:
   • BuildDefinition() — constructs and resolves a ParserDefinition from generated code.
   • Build() — convenience wrapper that currently resolves the result of BuildDefinition() again.
   • Grammar — eagerly initialized, process-wide cached CompiledGrammar property.
   • Tokenize(...) and conservative Parse(...) — delegate to Grammar.
   • generated embedded-code policy/context members and ParseWithEmbeddedCode(...) overloads when applicable.

The generated class is a partial so you can add hand-written members alongside it.

---

Getting started

1 — Reference the generator

In your .csproj, reference the generator as an analyzer and the runtime as a normal library:

<ItemGroup>
  <ProjectReference Include="..\Utils.Parser\Utils.Parser.csproj" />
  <ProjectReference Include="..\Utils.Parser.Generators\Utils.Parser.Generators.csproj"
                    OutputItemType="Analyzer"
                    ReferenceOutputAssembly="false" />
</ItemGroup>

Or with NuGet packages:

<ItemGroup>
  <PackageReference Include="omy.Utils.Parser" Version="2.0.0-rc.1" />
  <PackageReference Include="omy.Utils.Parser.Generators" Version="2.0.0-rc.1"
                    OutputItemType="Analyzer"
                    ReferenceOutputAssembly="false" />
</ItemGroup>

2 — Declare the grammar file

<ItemGroup>
  
  <CompilerVisibleItemMetadata Include="AdditionalFiles" MetadataName="Namespace" />
  <CompilerVisibleItemMetadata Include="AdditionalFiles" MetadataName="ClassName" />

  <AdditionalFiles Include="Parser\Exp.g4">
    <Namespace>MyApp.Parser</Namespace>
    <ClassName>ExpGrammar</ClassName>
  </AdditionalFiles>
</ItemGroup>

Metadata	Default	Description
Namespace	(empty — global namespace)	C# namespace for the generated class.
ClassName	File name without extension	Name of the generated partial static class.

3 — Add a hand-written partial stub (optional)

The generator emits the full implementation. A stub in your source tree lets the IDE discover the type even before the first build:

// Parser/ExpGrammar.cs
namespace MyApp.Parser;

internal static partial class ExpGrammar { }

4 — Use the generated class

using MyApp.Parser;
using Utils.Parser.Runtime;

// Process-wide cached CompiledGrammar.
var grammar = ExpGrammar.Grammar;

// Or build a fresh resolved ParserDefinition.
var definition = ExpGrammar.Build();
var tokens = grammar.Tokenize("1 + 2 * 3");
var tree = grammar.Parse("1 + 2 * 3");

---

Generated code shape

Given the grammar file Exp.g4 with metadata Namespace=MyApp.Parser and ClassName=ExpGrammar, the generator emits roughly:

// <auto-generated/>
// Source: Exp.g4
namespace MyApp.Parser;

[global::System.CodeDom.Compiler.GeneratedCodeAttribute("Utils.Parser.Generators", "...")]
internal static partial class ExpGrammar
{
    /// <summary>Gets the compiled grammar cached for the generated facade.</summary>
    public static global::Utils.Parser.Runtime.CompiledGrammar Grammar { get; }
        = new global::Utils.Parser.Runtime.CompiledGrammar(Build());

    /// <summary>Builds and validates the grammar definition.</summary>
    public static global::Utils.Parser.Model.ParserDefinition Build()
        => global::Utils.Parser.Resolution.RuleResolver.Resolve(BuildDefinition());

    /// <summary>Constructs and resolves the grammar definition from generated code.</summary>
    public static global::Utils.Parser.Model.ParserDefinition BuildDefinition()
    {
        // … generated Rule / RuleContent construction …
        return global::Utils.Parser.Resolution.RuleResolver.Resolve(
            new global::Utils.Parser.Model.ParserDefinition( … ));
    }

    public static global::System.Collections.Generic.IReadOnlyList<global::Utils.Parser.Runtime.Token> Tokenize(string input)
        => Grammar.Tokenize(input);

    public static global::Utils.Parser.Runtime.ParseNode Parse(string input)
        => Grammar.Parse(input);
}

BuildDefinition() currently constructs and resolves a new definition. Build() calls RuleResolver.Resolve(BuildDefinition()), so it resolves that result a second time. This documents the emitted behavior as it exists in 2.0.0-rc.1; whether the duplicate resolution should be removed is a separate functional concern. Grammar is the single initialized CompiledGrammar cached by the generated facade. Parse(...) is conservative. Supported generated hooks execute only through ParseWithEmbeddedCode(...) or a policy explicitly created and used by the caller.

The cached static Grammar owns one mutable LexerEngine and one mutable ParserEngine. Concurrent calls to the generated static Parse(...) or Tokenize(...) facade are therefore not safe. Concurrent consumers must synchronize access or create a separate CompiledGrammar instance for each concurrent operation. Execution contexts and reusable generated policies are mutable as well and must not be shared concurrently.

See the normative 2.0.0-rc.1 production support contract for the guaranteed generator subset. In particular, direct and transitive imports are emitted from the common composition plan: effective parser/lexer rules, fragments, modes, declared tokens, and channels execute from the generated facade. tokenVocab remains lexer-only, the entry owns the root/options/actions, and APU0107 covers only uniquely resolved effective parser targets. Qualified alias calls remain unsupported.

Until the release workflow publishes a versioned RC directory, use the current latest API documentation. A version-specific link will be added when the corresponding documentation artifact is deployed.

Generated-C# rule-call arguments

The emitter contains a narrow automatic binding path for positional simple-literal rule-call arguments. The shipped generator exposes an explicit project-wide opt-in, disabled by default. Normal package consumers get metadata/helper behavior only until they enable the MSBuild property:

<PropertyGroup>
  <UtilsParserEnableGeneratedRuleArgumentBinding>true</UtilsParserEnableGeneratedRuleArgumentBinding>
</PropertyGroup>

The NuGet build assets declare CompilerVisibleProperty, so projects that reference the generator package do not need to add that item manually. The option is project-wide and participates in generator validation and emission; changing only this property reuses unchanged per-file parsing and then regenerates the affected project outputs. With the option enabled, the generated parser can bind calls such as:

grammar P;

start : child[42] ;

child[int value]
@init {
    var v = GetRequiredRuleParameter<int>(context, "value");
}
    : A
    ;

A : 'a';

When UtilsParserEnableGeneratedRuleArgumentBinding is true, automatic binding is active only on generated-C# overloads that do not accept a caller basePolicy:

P.ParseWithEmbeddedCode(input);
P.ParseWithEmbeddedCode(input, executionContext);

With the option enabled, those overloads require exact positional arity, convert supported simple literals to allowlisted declared parameter types, and submit one managed seed batch before the child rule is entered. Invalid generated binding can throw at runtime before a child lifecycle hook observes partial seeds. The conservative Parse(...) overload is unchanged: it builds syntax only and does not execute generated hooks or bind rule-call arguments.

The overload that accepts a caller policy intentionally preserves the caller's rule-call policy instead of composing the generated automatic binding wrapper, even when the generation flag is enabled:

P.ParseWithEmbeddedCode(input, executionContext, basePolicy);

Limits are deliberate: positional arguments only, simple literals only, allowlisted target types only, no omitted/default argument consumption, no named or mixed arguments, no arbitrary expressions or method calls, and no ANTLR-style generated rule signatures.

---

Diagnostics

ID	Severity	Meaning
APU0100	Error	The .g4 file could not be parsed or the C# could not be emitted. The error message includes the file name and the underlying exception message.
APU0106	Warning	UtilsParserEnableGeneratedRuleArgumentBinding was not true or false; generation continues with generated rule-argument binding disabled.

---

Comparison with runtime parsing

	Antlr4GrammarConverter.Parse() (runtime)	Utils.Parser.Generators (build time)
When grammar is parsed	At runtime, on first call	At compile time
Startup cost	Parses + resolves the .g4 each run	Zero — grammar is pre-built C#
Deployment	Ships the .g4 file with the binary	Only the generated .cs is needed
Suitable for	Prototyping, dynamic grammars	Production, libraries, tight startup budgets

---

License

Apache 2.0 — see the repository root for details.

Embedded C# parser code

For parser semantic predicates and inline parser actions, the generator emits executable C# hooks in addition to preserving the raw model metadata strings. This is the source-generator C# opt-in path documented in docs/parser/ANTLRCompatibility.md. It is intentionally separate from Utils.Parser.Expressions and does not use IExpressionCompiler: embedded C# is compiled by Roslyn together with the consuming project.

Embedded parser code is target-language code. The standard source generator emits embedded code through NoOpParserEmbeddedCodeTransformer, so grammar code appears in generated C# as written except for normal wrapping and indentation. Custom transformer injection is currently available through direct emitter/runtime preparer APIs; the standard source-generator discovery path uses the no-op default. Transformed embedded C# is written into generated source by the internal CSharpEmbeddedCodeInjector, which centralizes generated markers, line-ending normalization, and four-space indentation for named-action regions and generated hook bodies. Parser and lexer named-action emission is selected through internal NamedActionInjectionDescriptor values so the six generated-C# header/member/footer families share one transform-and-inject path while preserving their existing regions and source positions. The injector receives only TransformedEmbeddedCode; transformation and diagnostic validation remain owned by the existing transformer boundary. Parser and lexer inline hook metadata now uses one internal EmbeddedCodeHook representation with explicit parser/lexer and predicate/action discriminants. Hook collection is shared by the internal EmbeddedHookCollector, while private parser and lexer strategies keep priority ordering, left-recursive parser roots, lexer modes, index sentinels, method-name prefixes, and transformation locations explicit.

Preferred no-transformer embedded code should call the generated helper APIs directly instead of relying on ANTLR-style $... convenience syntax:

rule
@after {
    SetRuleReturn(context, "value", 42);
}
    : A ;

Useful direct C# helper style includes:

var value = GetRequiredRuleParameter<int>(context, "value");
SetRuleLocal(context, "total", value);
SetRuleReturn(context, "result", value + 1);
var childValue = GetRequiredLabeledRuleCallReturn(context, "x", "value");
// In generated-C# inline parser actions and @after, x=child may also use $x.value sugar.
var values = GetLabeledRuleCallReturns(context, "xs", "value");

Supported forms include expression-bodied parser predicates, block-bodied parser predicates with return, multi-line predicate blocks with local variables and return, single- or multi-statement inline parser actions, rule @init / @after lifecycle hooks, parser @header / @parser::header C# source-file injection, parser @members / @parser::members execution-context injection, and parser @footer / @parser::footer trailing source injection, for example:

grammar P;
start : {
    var isStart = inputPosition == 0;
    return isStart && ruleName == "start";
}? {
    OnBefore(context);
    OnAfter(context);
} A ;
A : 'a' ;

Parser @header and @parser::header blocks are emitted verbatim near the top of the generated C# file before generated type declarations, allowing ordinary C# header content such as using directives to compile. Parser @members and @parser::members blocks are emitted verbatim into {ClassName}ExecutionContext. Parser @footer and @parser::footer blocks are emitted verbatim near the end of the generated file after generated type declarations as trailing generated C# source; this footer location is not a second header region and should not be documented or used as a place where using directives are valid. These are source-generator C# compatibility bridges only; invalid C# is reported by Roslyn and this does not imply full ANTLR target-language compatibility.

The generated static facade exposes ParseWithEmbeddedCode(string input) for default opt-in execution and CreateRuntimePolicy({ClassName}ExecutionContext executionContext, ParserRuntimeFeaturePolicy? basePolicy = null) for explicit policy binding. The existing generated Parse(string input) helper continues to use the default conservative runtime policy and does not execute generated embedded-code hooks. ParseWithEmbeddedCode(string input) creates a new {ClassName}ExecutionContext instance for that call, and ParseWithEmbeddedCode(string input, {ClassName}ExecutionContext executionContext) lets advanced callers provide an explicit context. CreateRuntimePolicy({ClassName}ExecutionContext executionContext, ParserRuntimeFeaturePolicy? basePolicy = null) binds the returned policy to the supplied context; reusing that policy intentionally reuses that context state, and no generated CreateRuntimePolicy() overload creates a hidden context. The context is a generated parser execution context, not a lexer mode or lexer-state context. Generated predicate/action hooks are instance methods on that context. Generated execution contexts also expose internal Fork(), CopyFrom({ClassName}ExecutionContext source), and GetExecutionStateKey() helpers, plus explicit private rule-local frame helpers (GetRuleLocal, TryGetRuleLocal, SetRuleLocal, and GetRuleLocalDescriptors) and rule-return frame helpers (GetRuleReturn, TryGetRuleReturn, SetRuleReturn, and GetRuleReturnDescriptors) that generated lifecycle hook bodies can call through their ParserRuleLifecycleContext. Before a generated @init hook runs, the generated lifecycle executor allocates every named declaration from locals [...] as a missing-only context.InvocationFrame.Locals entry whose value is null; existing frame values are preserved. The rule-local helpers read and write only context.InvocationFrame; allocation does not infer C# types, instantiate arrays or value-type defaults, generate typed local fields/properties, or expose locals as implicit action variables. The rule-return helpers also read and write only the active context.InvocationFrame; return entries are not auto-allocated, not typed, not exposed as implicit variables, and not propagated automatically; direct helper calls project the same frame state as object?; optional C# ANTLR-style $... rewrites, when explicitly enabled through a transformer, target those helpers. Fork() delegates to Utils.Parser.Runtime.ParserExecutionContextCopier<TContext>.Copy(...), including ICloneable precedence when a user partial context implements it; CopyFrom(...) validates source and delegates to ParserExecutionContextCopier<TContext>.CopyTo(source, this). ParserEngine uses the generated execution-state manager for managed execution-context rollback around parser backtracking attempt boundaries in generated C# opt-in paths; no action buffering or replay is active, and the copy semantics remain the copier's shallow structural semantics (collection interfaces and known concrete collections are copied, dictionary/set comparers are preserved, and no deep clone is attempted). Predicate code without a return keyword is emitted as return <expression>;; predicate code that contains return is emitted as statements inside the generated bool hook. Action code is emitted as statements inside a generated void hook, so multi-instruction and multi-line actions can use generated locals such as context, ruleName, inputPosition, alternativeIndex, elementIndex, and actionCode. Predicate hooks similarly expose predicateCode. Action code can call members injected into or supplied by another declaration of the generated execution-context partial class, including member declarations that rely on parser-header using directives. Invalid embedded C# is reported as a normal C# compilation error by Roslyn; the generator does not parse C# semantically. Hook dispatch keys are aligned with the runtime ParserEngine indexes for single-item alternatives, sequences, quantified content, negation predicate probes, equal source text in multiple alternatives, and direct-left-recursive tail views because generated helpers resolve the generated definition before parsing with the generated policy. Runtime-inline lexer actions and predicates, unsupported grammar actions, parser actions outside inline alternative positions, rule parameters/returns/exceptions execution, action buffering/replay, and arbitrary parser state mutation are not executed by this support. Simple lexer inline actions and predicates are executed only by generated-C# opt-in policies. Combined and lexer grammars may inject grammar-level @lexer::header, @lexer::members, and @lexer::footer as generated-C# compatibility blocks with dedicated lexer markers; parser-only grammars keep @lexer::* unsupported because no lexer is generated. Simple lexer inline action and predicate execution is limited to generated-C# opt-in; conservative Parse(...) remains unchanged.

Design note: docs/parser/RuleArgumentsAndReturnsPlan.md tracks the future generated-C# opt-in integration plan for rule arguments, parameters, returns, labels, rollback, and memoization. The helpers documented here are explicit manual building blocks; they are not automatic ANTLR callee[expr] evaluation, automatic parameter binding, ANTLR-compatible generated parser signatures, or a full ANTLR attribute model.

Rule-call argument syntax callee[...]

Rule-call argument clauses such as child[42] are recognized by the generator's G4 parser and preserved as raw metadata text on the emitted RuleRef (RawArguments property, outer brackets excluded). Reported with UP1037 RuleCallArgumentsPreservedAsMetadata.

At runtime, the raw argument text is also carried into ParserRuleCallResult.RawArguments on the parent frame's last completed child call result. Generated C# opt-in code can inspect it explicitly:

start @after {
    Raw = GetLastRuleCallResult(context)?.RawArguments;
    // or:
    TryGetLastRuleCallRawArguments(context, "child", out string? rawArgs);
}
    : child[42] ;

This is metadata only by default: the argument text is not evaluated, parsed as C# expressions, or bound to child rule parameters unless the caller explicitly installs one of the limited positional or named literal policies described below. Call-site metadata is rollback-safe and memoization-safe. PendingChildSeeds, InvocationFrame.Parameters, generated Parse(...), and generated rule method signatures are unchanged.

For named argument forms (value: 42, value = 42), use the named helpers:

// In an inline action between child[value: 42, text: "hello"] and child2:
if (TrySplitLastRuleCallNamedRawArguments(context, "child", out var named))
    SetNextRuleParametersFromNamedRawArguments(context, "child2", named,
        new ParserRawNamedArgumentParameterMapping { ParameterName = "value", ArgumentName = "value", Map = s => int.Parse(s) },
        new ParserRawNamedArgumentParameterMapping { ParameterName = "text",  ArgumentName = "text",  Map = s => s.Trim('"') });

Missing argument name returns false; no partial seeding. Duplicate ParameterName: last wins.

To map multiple positional slices in one call, use SetNextRuleParametersFromRawArguments:

if (TrySplitLastRuleCallRawArguments(context, "child", out var args))
    SetNextRuleParametersFromRawArguments(context, "child2", args,
        new ParserRawArgumentParameterMapping { ParameterName = "value", Index = 0, Map = s => int.Parse(s) },
        new ParserRawArgumentParameterMapping { ParameterName = "text",  Index = 1, Map = s => s.Trim('"') });

Validates all indices before applying any seed. Last mapping wins for duplicate names. Mapper exceptions propagate.

To split raw argument text into individual top-level slices and then seed explicitly:

// In an inline action between child[42, "hello"] and child2:
if (TrySplitLastRuleCallRawArguments(context, "child", out var args))
{
    SetNextRuleParameterFromRawArguments(context, "child2", "value", args[0], s => int.Parse(s));
    SetNextRuleParameterFromRawArguments(context, "child2", "text",  args[1], s => s.Trim('"'));
}

Splitting respects nested (), [], {}, and quoted strings. Syntactic only — no argument is evaluated. Backed by Utils.Parser.Runtime.ParserRawArgumentSplitter.SplitTopLevel.

To map raw argument text into a future child seed explicitly, use the helper:

// In an inline action between child[42] and child2:
if (TryGetLastRuleCallRawArguments(context, "child", out string? raw))
    SetNextRuleParameterFromRawArguments(context, "child2", "value", raw, s => int.Parse(s));

This helper requires an explicit mapper delegate and never evaluates arguments automatically. Null rawArguments returns false. Mapper exceptions propagate. Seeds the next invocation of the named rule.

Use SetNextRuleParameter(...) for direct explicit seeding. Bare $param reads are supported only inside generated embedded C# for the current rule and do not seed child calls.

Rule-reference label metadata (x=child, xs+=child)

Rule-reference labels are recognized and preserved as passive metadata by both the ANTLR converter path and the source-generator G4 parser path:

• x=child sets LabelKind = Assignment; xs+=child sets LabelKind = List; unlabeled references have LabelKind = None.
• Label metadata is stored on RuleRef.Label / RuleRef.LabelName / RuleRef.LabelKind in the model; GrammarEmitter emits Label: new RuleLabel(...) in generated BuildDefinition().
• At runtime, ParserEngine calls AnnotateLastChildCallLabel after each successful child rule completion so the call-site label is visible via ParserRuleCallResult.LabelName and ParserRuleCallResult.LabelKind on the parent frame.
• Labels compose with callee[...] raw arguments: both can coexist on the same rule reference.
• Label metadata is rollback-safe and memoization-safe.
• Generated C# opt-in code can inspect label metadata explicitly:

start @after {
    var r = GetLastRuleCallResult(context);
    string? label = r?.LabelName;          // "x", "xs", or null
    string? kind  = r?.LabelKind.ToString(); // "Assignment", "List", or "None"
}
    : x=child ;

Labels remain explicit managed metadata: no bare $x, $xs, $x.value, $xs.value, implicit label variables, typed label fields/properties, automatic parse-node storage, automatic binding, automatic argument evaluation, automatic parameter seeding, or generated parser method signatures are added. Labels on non-rule-reference elements (literals, groups, etc.) emit diagnostic UP1022 LabelOnNonRuleReferenceIgnored. Conservative Parse(...) remains conservative; hooks do not execute and label metadata is not exposed. No lexer label support is added.

Shared runtime metadata alignment

Generated runtime dispatchers for parser predicates, parser actions, lexer predicates, and lexer actions are emitted by one internal dispatcher emitter. Immutable descriptor data carries the four domain configurations (interface, context type, return type, code property, invocation arguments, success expression, and fallback expression), while the wrappers remain explicit and short. This is an internal refactor only: generated class names, method signatures, comparison order, hook order, success outcomes, fallbacks, and public APIs are preserved. Generated hook methods are emitted by one internal method emitter. Immutable method descriptors carry the four domain configurations (owner, kind, XML summary, return type, parameters, context-local profile, and predicate/action body factory), while the explicit wrappers remain short. Parser predicate/action locals stay parser-only, lexer hooks do not gain parser locals, lexer actions keep LexerActionExecutionResult result, and CSharpEmbeddedCodeInjector remains the central body injector for transformed embedded code.

Generated C# hooks for parser semantic predicates and inline parser actions continue to be collected from the generator's G4Grammar AST because the analyzer package targets netstandard2.0 and does not reference the net8.0 runtime model assembly. The hook collector is intentionally kept aligned with Utils.Parser.EmbeddedCode.EmbeddedCodeRuntimeDiscovery: it uses the same runtime index rules for priority-ordered alternatives, single-item alternatives, sequences, quantifier inner parsing, negation probes, duplicate source text, and direct-left-recursive base/tail alternatives. Unit tests compare generated hook names against shared ParserDefinition discovery metadata for sensitive dispatch cases.

@header and @parser::header blocks are parser header code in parser or combined grammars; supported blocks are injected verbatim near the top of the generated C# source in grammar source order and report warning UP1035 EmbeddedHeaderInjectedByGenerator; Roslyn reports invalid C#. @parser::members blocks, and legacy unscoped parser @members blocks in parser or combined grammars, are injected verbatim into the generated parser execution context/class in grammar source order and report warning UP1031 EmbeddedMembersInjectedByGenerator; parser inline actions and supported lifecycle hooks can call those members. Roslyn reports invalid C# and member-name collisions. @parser::footer blocks, and legacy unscoped parser @footer blocks in parser or combined grammars, are injected verbatim as trailing generated C# source near the end of the generated parser file in grammar source order and report warning UP1036 EmbeddedFooterInjectedByGenerator; Roslyn reports invalid C#. In combined or lexer grammars only, @lexer::header, @lexer::members, and @lexer::footer mirror those source-generation injection points with dedicated lexer markers; @lexer::members is emitted into the existing generated execution context and is not a separate lexer runtime type. Parser-only grammars keep @lexer::* unsupported because no lexer is generated. The footer injection point is not a second header region, so the documentation does not claim using directives are valid there. Unsupported constructs are not promoted to executable generated hooks or injected compatibility blocks. Visible unsupported embedded-code constructs in the generator AST report warning UP1029 EmbeddedCodeConstructNotExecutedByGenerator with deterministic wording, including unsupported grammar actions, parser named actions in lexer grammars, lexer-grammar unscoped @header / @members / @footer, parser-grammar scoped @lexer::*, unknown parser/lexer named-action names, and unknown named-action scopes. Invalid C# inside a supported parser predicate/action/lifecycle hook or injected parser/lexer header/member/footer block remains a Roslyn compilation error rather than a custom unsupported-construct diagnostic. The default no-op embedded-code transformer preserves parser and lexer named-action content unchanged; optional transformers remain opt-in, and ANTLR-style $... current-rule rewriting is not applied to parser or lexer header/member/footer content. This alignment adds only explicit generated-C# opt-in simple lexer action/predicate execution, does not claim full ANTLR target-language compatibility, and does not change ParserEngine or the default Parse(...) behavior.

Explicit parser rule-call execution policy

Generated C# preserves ParserRuntimeFeaturePolicy.RuleCallExecutionPolicy from the caller-supplied basePolicy. A custom IParserRuleCallExecutionPolicy can therefore observe BeforeRuleCall(...) and AfterRuleCall(...) through either CreateRuntimePolicy(executionContext, basePolicy) or the generated ParseWithEmbeddedCode(input, executionContext, basePolicy) overload. Existing CreateRuntimePolicy(...) and ParseWithEmbeddedCode(...) overloads remain available, and generated Parse(...) remains conservative.

The default policy is NullParserRuleCallExecutionPolicy.Instance. The callback context exposes passive current-call-site metadata, including raw arguments and label name/kind, and exposes the annotated completed result after a successful tracked child call. This does not execute callee[...], evaluate arguments, bind parameters, set pending seeds automatically, generate typed parameters/returns or label variables, or support $param, $x, $x.value, or $rule.value. External side effects performed by a custom policy are not automatically rolled back; only separately managed rollback-aware parser state participates in capture/restore. Raw argument and label annotations remain current-call-site safe after rollback and memoization.

Opt-in positional literal rule-call binding

Generated parsers can use PositionalLiteralRuleCallExecutionPolicy only through the existing caller-supplied basePolicy path:

var basePolicy = ParserRuntimeFeaturePolicy.Default with
{
    RuleCallExecutionPolicy = new PositionalLiteralRuleCallExecutionPolicy()
};

P.ParseWithEmbeddedCode(input, executionContext, basePolicy);

The default and generated Parse(...) remain metadata-only/conservative. The policy requires exact positional arity and binds declared parser-rule parameter names without enforcing their C# declaration types. It supports only null, lowercase Booleans, signed decimal int/long, finite invariant double, quoted strings, and character literals with a small escape set. Named binding, arbitrary expressions, Roslyn evaluation, $param writes/chains, labels/returns as call arguments, and lexer execution are not supported. Managed pending seeds are applied as one all-or-none batch, are rollback-aware, and generated memoization distinguishes the supported literal values deterministically. Existing explicit helpers continue to accept arbitrary values: deterministic scalars and IParserExecutionStateHashable values receive stable keys, while other objects force volatile keys that bypass completed-result reuse while pending.

Opt-in named literal rule-call binding

Generated parsers can separately install NamedLiteralRuleCallExecutionPolicy through the same caller-supplied basePolicy path:

var basePolicy = ParserRuntimeFeaturePolicy.Default with
{
    RuleCallExecutionPolicy = new NamedLiteralRuleCallExecutionPolicy()
};

P.ParseWithEmbeddedCode(input, executionContext, basePolicy);

This policy is not the default and is not automatically combined with positional binding. It consumes the generated runtime's existing NamedRawArguments metadata for both name: literal and name = literal. Names match declared parser-rule parameters with StringComparer.Ordinal; argument order does not matter, but exact coverage is required. Missing, extra, case-mismatched, blank, or duplicate declared names fail the whole call. Optional/default parameters, partial binding, and mixed positional/named syntax are unsupported. Duplicate raw names inherit the splitter's documented last-wins result.

All values must be accepted by ParserSimpleLiteralParser. Declared C# types are not checked or converted, and arbitrary expressions are not evaluated. One complete atomic pending-seed batch is applied only after validation, preserving rollback and deterministic memoization behavior for supported values. Generated Parse(...) remains conservative. No $param, $x, $x.value, $rule.value, return/label binding, or lexer support is added.

Explicit typed literal rule-call binding

Generated opt-in parsing can preserve a caller-supplied typed call policy through basePolicy:

var basePolicy = ParserRuntimeFeaturePolicy.Default with
{
    RuleCallExecutionPolicy = new TypedPositionalLiteralRuleCallExecutionPolicy()
};

var result = GeneratedGrammar.ParseWithEmbeddedCode(input, executionContext, basePolicy);

Use TypedNamedLiteralRuleCallExecutionPolicy separately for ordinal-name name: literal or name = literal calls. Neither typed policy is installed automatically, they are not combined, and generated Parse(...) remains conservative. The existing PositionalLiteralRuleCallExecutionPolicy and NamedLiteralRuleCallExecutionPolicy remain untyped.

Typed policies recognize only the C# aliases bool, byte, sbyte, short, ushort, int, uint, long, ulong, float, double, decimal, char, string, object, their exact canonical System.* names, and a single nullable suffix. They use checked integral conversion, exact-preserving integral-to-floating and double-to-float conversion, integral-to-decimal conversion, and limited string/character conversion. Null is accepted only for reference targets and nullable value targets. Strings are not parsed into numbers or Booleans; floating-point-to-integral conversion is rejected.

Parser-rule parameter descriptors preserve top-level default text as passive RawDefaultValue metadata. Only the typed policies consume it. Typed positional calls may omit trailing parameters when each omitted declaration has a supported simple-literal default; typed named calls may omit parameters in any order under the same condition. Explicit arguments override defaults, and unused invalid defaults are not evaluated. Existing untyped policies continue to require exact arity or exact name coverage and ignore defaults.

All explicit values and required defaults finish parsing and conversion before one rollback-managed atomic seed batch. Memoization keys use the final converted effective runtime values and preserve runtime type distinctions; explicit and default forms producing identical state may share memoized results. No arbitrary type resolution, arrays, generics, enums, user-defined types, Roslyn conversion, general default-expression evaluation, parameter references, constants, member access, calls, default, nameof, interpolation, $param forms, return/local/label binding, or lexer argument/action/predicate execution is provided. Generated Parse(...) remains conservative.

Explicit labeled child-call results

Generated embedded-code opt-in contexts provide generic lifecycle and inline-action helpers for parser rule-reference labels:

bool found = TryGetLabeledRuleCallReturn(context, "x", "value", out object? value);
IReadOnlyList<ParserRuleCallResult> calls = GetLabeledRuleCallResults(context, "xs");
IReadOnlyList<object?> values = GetLabeledRuleCallReturns(context, "xs", "value");

x=child retains the last successful immutable ParserRuleCallResult; xs+=child appends successful results in execution order. Child returns are captured after child @after. Missing return keys and present-null values remain distinct. List return projection includes present-null entries and skips calls where the key is absent. Managed snapshots make retention rollback-safe, and memoized child results receive the current call site's label before binding. Assignment and list namespaces are separate if a grammar reuses one lexical label with both operators.

This remains metadata-driven access. The generator does not emit bare $x/$xs, implicit variables, typed label fields, typed return accessors, automatic return assignment, or lexer label/return support. Positional, named, typed, and default-aware argument policies are unchanged, and generated Parse(...) remains conservative.

Direct helper APIs for parser-managed state

With the default no-op transformer, embedded parser code should be ordinary C# that calls generated helper methods explicitly. This keeps the parser/generator core language-neutral and keeps rollback-aware state access visible in the source text.

start
@after {
    var childValue = GetRequiredLabeledRuleCallReturn(context, "x", "value");
    SetRuleReturn(context, "value", childValue);
}
    : x=child
    ;

child returns [int value]
@after {
    SetRuleReturn(context, "value", 42);
}
    : A
    ;

Existing explicit helpers remain available for plain C# embedded code, including rule parameter, rule local, rule return, last child-call, assignment-label, and list-label result access. Generated Parse(...) remains conservative; embedded-code hooks execute only through the generated opt-in policy helpers.

Optional C# ANTLR-style transformer

ANTLR-style convenience forms such as $x.value, $xs.value, $rule.value, $param, and $local are not core parser syntax and are not rewritten by default. With NoOpParserEmbeddedCodeTransformer, they are emitted unchanged. If unchanged $... text is not valid C#, the consuming project compilation fails normally.

A C#-specific optional compatibility transformer may rewrite narrow ANTLR-style convenience syntax to generated helper calls. That transformer is not language-neutral, is not required by parser core, and does not imply full ANTLR embedded action compatibility. Its limitations do not affect parser core behavior, and future full code transformers must remain behind IParserEmbeddedCodeTransformer.

Embedded parser code transformation

Generated C# preserves embedded parser code by default through NoOpParserEmbeddedCodeTransformer. The generator does not treat ANTLR-style $... parser attributes as intrinsic grammar semantics. If a project needs target-language conveniences, it must opt into an IParserEmbeddedCodeTransformer through APIs that accept one; generated hook bodies emit the transformed code, and transformer errors stop safe generation. Existing generated helper methods remain available for plain C# embedded code.

Generated/source-generator emission applies the embedded-code transformer before emitting parser @header, parser @footer, rule @init, rule @after, inline parser actions, and semantic predicates where those locations are supported. The standard source generator uses the no-op transformer; direct emitter APIs can supply a custom transformer for tests or specialized tooling.

Internally, collection produces RawEmbeddedCode, which crosses the shared transformation-and-validation boundary exactly once with its strongly typed context. The resulting TransformedEmbeddedCode is classified as a predicate expression/fragment or action fragment by GeneratedEmbeddedCodeBody and written only by CSharpEmbeddedCodeInjector. Raw hook text remains available for diagnostics and auditing but is never consumed by a C# emitter, and a hook that has not completed transformation is rejected before any generated source is written. This generator-specific tail does not construct runtime expressions or invoke an expression compiler.

Optional C# transformer lexer action attributes

Generated C# can opt into a narrow lexer inline-action attribute rewrite set through CSharpAntlrStyleParserEmbeddedCodeTransformer. The default/no-op transformer preserves lexer $... syntax unchanged. This rewrite applies only to lexer inline actions, not parser actions, lexer predicates, or runtime-inline execution.

Supported lexer action reads are $text, $type, $channel, $mode, $line, and $pos. $text, $type, $channel, and $mode read passive LexerActionExecutionContext.Text, TokenType, Channel, and Mode values. $line is rewritten to GetRequiredLexerLine(context) and reads LexerActionExecutionContext.Line. $pos is rewritten to GetRequiredLexerPos(context) and reads LexerActionExecutionContext.Column. Both values come from SourceSpan and identify the 1-based beginning of the accepted token/chunk; $pos is therefore this runtime's 1-based source column, not full ANTLR charPositionInLine compatibility.

Generated-C# opt-in lexer inline actions also support a deliberately narrow write subset: simple $type = identifierOrString;, $channel = identifierOrString;, and $mode = identifierOrString; statements. The transformer rewrites those statements to SetLexerType(result, "..."), SetLexerChannel(result, "..."), and SetLexerMode(result, "..."), which set bounded LexerActionExecutionResult.TokenType, LexerActionExecutionResult.Channel, and LexerActionExecutionResult.Mode fields. Generated hooks do not mutate Token directly. LexerEngine applies accepted action-result token and mode mutations before lexer commands. Commands remain authoritative: type(...), channel(...), mode(...), pushMode(...), popMode, skip, and more keep their existing language-neutral behavior. $mode = ... replaces the current mode like mode(...); it does not push or pop modes, and pushMode(...)/popMode keep their stack semantics. Reads such as $type, $channel, and $mode continue to read the passive LexerActionExecutionContext; writes do not retroactively change that read context inside the same action body.

Unsupported forms include $text = ..., $line = ..., $pos = ..., compound/coalescing/increment writes such as $mode += ..., $type += ..., $channel ??= ..., $mode++, $type++, $channel++, reads or writes of lexer $... attributes inside lexer predicates, complex expression writes, ref/out writes, chained attributes, runtime-inline lexer execution, and a separate runtime lexer.

Lexer action transaction boundary

Lexer predicates execute while the engine explores a recognition path. They may allow or reject only that path and have no supported lexer-attribute write surface. This prevents runtime-managed predicate mutations, but does not undo arbitrary external effects in user code.

Lexer actions on rejected paths are not executed. Once a token is selected, its actions share one fresh LexerActionExecutionResult local to that acceptance and execute before commands. Repeated assignments use the existing last-write-wins behavior. The engine applies requested TokenType, Channel, and Mode values before type(...), channel(...), skip, more, mode(...), pushMode(...), and popMode; those commands remain authoritative. $mode = ... replaces the current mode like mode(...), without push/pop semantics.

Lexer-owned operational state is split by lifetime: persistent LexerEngine fields cover the current mode, mode stack, more accumulation, and more start position; a tokenization session owns the TextReaderBuffer input position, emitted-token collection, and per-call extension invocation contexts; recognition and acceptance own best-match data, collected commands and action occurrences, token/chunk construction data, and the accepted token's LexerActionExecutionResult. None of these categories belongs to IParserExecutionStateManager. No general lexer snapshot manager or parser/lexer manager selected by an isLexer flag exists. Mutable @lexer::members state and effects on I/O, shared services, or external objects are not generally rollback-safe; skip, later exceptions, and parser failure do not reverse them. Any future lexer state-manager contract requires a concrete need and a separate PR.

Optional ANTLR-style local writes

The default/no-op embedded-code transformer preserves $local = ... and every other $... fragment unchanged. ANTLR-style writes to current-rule locals are supported only when generated C# explicitly opts into CSharpAntlrStyleParserEmbeddedCodeTransformer. The transformer rewrites simple assignment, compound assignment, and standalone prefix/postfix increment/decrement to typed SetRequiredRuleLocal<T>(...) calls using the raw local declaration type. Compound assignment is emitted as getter/operator/setter and does not emulate C# compound-assignment narrowing conversions; grammar authors must write explicit casts when needed. Parameters, returns, labels, list-label projections, token labels, lexer attributes, ref/out, nested assignments, and increment/decrement expression values remain unsupported. Direct helper APIs such as SetRuleLocal(...) / SetRequiredRuleLocal<T>(...) remain the preferred no-transformer C# style. Future richer action conveniences must remain isolated behind IParserEmbeddedCodeTransformer.

Optional C# transformer current-rule return writes

The optional C# ANTLR-style transformer supports a narrow current-rule return write convenience syntax in rule @after code and inline parser actions. The default no-op transformer preserves bare $returnName = ... text unchanged. This is a logic-stage source rewrite only: broader ANTLR return semantics and parent convenience access remain separate work.

Supported forms use the bare return attribute declared by the current rule in @after and inline parser actions, for example $value = 42;, compound assignments such as $value += 1;, and standalone increment/decrement statements. The transformer rewrites those forms to explicit typed helper calls such as SetRequiredRuleReturn<T>(context, "value", ...) and GetRequiredRuleReturn<T>(context, "value"). These helpers write the parser-managed current-rule invocation frame. The default no-op transformer still preserves $returnName = ... unchanged.

Parameters, child return access such as $child.value, labeled rule-call returns such as $c.value, list-labeled projections, lexer attributes, ref/out, semantic predicates, @init, and dotted current-rule return attributes such as $rule.value and $rule.value = ... remain unsupported in this PR. Use bare $returnName = ... only for declared current-rule return attributes when opting into the C# transformer.

Generated-C# explicit simple positional rule-call binding

Generated parsers can explicitly install a generated-C#-only rule-call policy for ParseWithEmbeddedCode(...) when generation enables simple positional rule-argument binding. When a parser rule call supplies raw positional arguments, the generated policy first requires the raw positional argument count to exactly match the declared target-rule parameter count, including zero-parameter target rules; an explicit empty argument list such as child[] is therefore valid only when the target declares zero parameters. This generated-C# automatic boundary is stricter than the reusable typed runtime policy: declared parameter defaults are not consumed to satisfy omitted generated-C# call-site arguments. After exact arity passes, the generated policy converts supported simple literals and submits one atomic managed seed batch to the existing invocation-frame parameter store. The conservative generated Parse(...) path remains unchanged and does not execute this binding path.

Supported automatic generated-C# argument forms are intentionally narrow: exact-arity simple positional literals that the typed literal binding policy can convert to the declared parameter type, including decimal integer literals for int parameters. Named arguments and arbitrary C# expressions remain unsupported and are rejected deterministically in the generated-C# explicit binding path before child lifecycle hooks can observe partially seeded state. Full ANTLR-compatible generated rule signatures such as child(int value) are still not emitted; generated hooks should continue to read parameters through frame helpers such as GetRequiredRuleParameter<T>(context, "name"), and the optional C# ANTLR-style transformer may rewrite $name to those helpers. Explicit runtime policies such as TypedPositionalLiteralRuleCallExecutionPolicy may still support simple typed defaults separately when callers install them directly.

The implementation uses existing parser-managed pending seeds, invocation frames, execution-state snapshots, rollback, and memoization boundaries. No target-language expression evaluator was added to ParserEngine.

Generated-C# returns/labels boundary and named-action strategy

The rule-return and labeled rule-call boundary follows the existing parser named-action architecture rather than a parallel implementation path. Classification of grammar-level named actions is centralized in EmbeddedMembersSupport: @members and @parser::members are parser compatibility blocks injected into the generated execution context, @header and @parser::header are injected near the top of generated C# source, and @footer and @parser::footer are injected as trailing generated C# source. Unsupported parser-scoped actions such as @parser::init and parser named actions inside lexer grammars remain deterministic diagnostics and are not generated-source injection points.

Parser embedded code must continue to pass through IParserEmbeddedCodeTransformer via TransformEmbeddedCode(...). The generated-C# path carries grammar fragments as RawEmbeddedCode, invokes ParserEmbeddedCodeTransformationService.TransformOrThrow(...), and emits only TransformedEmbeddedCode text into hook bodies. The default path preserves target-language code, and generated-C# embedded-code paths remain opt-in. Metadata is not execution authority: rule-return declarations may be present in grammar metadata, and labeled rule-call storage may be present in parser-managed frame state, but metadata/storage alone does not imply automatic runtime support, ANTLR-compatible label access, public typed parser contexts, $label.ctx, $ctx, or public ANTLR-style rule methods. Conservative Parse(...) remains unchanged, and ParserEngine remains target-language-neutral.

Future simple generated-C# return assignment/access should reuse generated execution-context helpers and optional transformer rewriting. Future labeled rule-call return access should build on existing labeled result storage where available. Any $... syntax support must be implemented through the parser embedded-code transformer, not the runtime parser core. No full ANTLR parser context model is promised by the current generated-C# compatibility bridge.

Explicit labeled return helper examples

Generated C# currently reads parent/labeled child returns with explicit helpers only. A child can write its current-rule return with bare $value when the optional C# transformer is enabled, but the parent should use helper calls:

grammar P;

@members { public int Seen = -1; }

start
@after {
    Seen = GetRequiredLabeledRuleCallReturn(context, "c", "value") is int v ? v : -1;
}
    : c=child ;

child returns [int value]
@after {
    $value = 42;
}
    : A ;

A : 'a' ;

List labels use GetLabeledRuleCallResults(context, "xs") and GetLabeledRuleCallReturns(context, "xs", "value"); absent list labels return empty lists. TryGetLabeledRuleCallReturn distinguishes missing returns from present-null values by returning true with value == null for present-null entries. Required helper calls throw deterministic parser attribute access exceptions for missing labels or return names. Failed alternatives are rolled back, and memoized child results keep child returns while applying the current successful call-site label. $c.value, $x.value, and $xs.value are supported only as generated-C# opt-in transformer sugar in inline parser actions and @after; $child.value, $rule.value, $ctx, $c.ctx, $xs.ctx, bare labels, token attributes, lexer attributes, typed contexts, public ANTLR-style parser methods, and general ANTLR attribute compatibility remain unsupported syntax.

Generated-C# list-label return sugar is intentionally narrow: $xs.value is available only in inline parser actions and @after when xs is a visible parser-rule list label from xs+=child and every referenced child rule declares value. The transformer rewrites only the $xs.value root/projection to GetLabeledRuleCallReturns(context, "xs", "value"); any following C# member access, for example .Count, is ordinary C#. The projection is read-only, reads only successful child calls, preserves order and present-null values, and follows the same rollback semantics as the explicit helper. It is unavailable in @init, semantic predicates, parser/lexer members, headers, footers, and lexer actions. $child.value, $rule.value, $xs.ctx, $ctx, typed parser contexts, public ANTLR-style parser methods, label writes, token attributes, and lexer attributes remain unsupported. Conservative Parse(...) remains unchanged and ParserEngine remains target-language-neutral.

Generated-C# parser return convenience boundary

Supported generated-C# opt-in convenience forms are deliberately narrow:

• bare $value reads/writes a declared return of the current rule;
• $c.value / $x.value read a declared child return through an assignment label such as c=child;
• $xs.value reads a list-label projection through xs+=child and returns the generated helper list.

The following remain unsupported and must produce deterministic transformer diagnostics rather than new runtime syntax: $child.value, $rule.value, $ctx, $c.ctx, $xs.ctx, bare $c / $xs label objects, writes to $c.value or $xs.value, label-return reads in @init, label-return reads in semantic predicates, token attributes such as $t.text, lexer attributes, typed parser contexts, public ANTLR-style parser rule methods, and general ANTLR attribute compatibility.

These forms are optional IParserEmbeddedCodeTransformer rewrites for generated C# only. The default/no-op transformer leaves $... text unchanged, conservative Parse(...) remains unchanged, and ParserEngine remains target-language-neutral. Parser-managed return and label state follows the existing rollback semantics; no rollback of external side effects is implied.

Generated rule-call argument binding diagnostics

When UtilsParserEnableGeneratedRuleArgumentBinding=true, the generator performs bounded static validation for generated-C# positional rule-call binding against uniquely resolved parser targets in the effective local/direct/transitive composition. Missing or ambiguous dependencies, imported-rule collisions, lexer-only targets, and unresolved calls do not produce APU0107. Aliased imports retain unqualified composition compatibility; qualified Alias.rule calls remain unsupported. This pass does not evaluate general expressions, does not change conservative Parse(...), and does not claim complete ANTLR4 delegate-grammar compatibility.

The validation mirrors the generated-C# contract: positional arguments only, supported simple literals only, exact arity, allowlisted declared parameter types, no default-value consumption, no named or mixed arguments, no arbitrary C# expression evaluation, no generated rule-signature changes, and no change to Parse(...). Incrementally, each .g4 file is parsed from its path, text, and own AdditionalFiles Namespace/ClassName metadata; unchanged parsed files can be reused. The generator still aggregates the parsed project with Collect() so import resolution, validation, and emission may rerun project-wide after a grammar or option change; this is not a promise of import-subgraph-only recalculation.

grammar P;
start : child[1 + 2] ;
child[int value] : A ;
A : 'a' ;

With the generated binding option enabled, this grammar reports APU0107 (Error) at the child[1 + 2] call site because 1 + 2 is an expression, not a supported simple literal. The invalid grammar file is not emitted, while other valid .g4 files in the same compilation continue to generate normally.

Shared import-composition plan boundary

The generator's G4Grammar model can now be adapted to the same Roslyn-free grammar composition planner used by Antlr4GrammarProjectCompiler. The adapter preserves G4Grammar, G4Rule, G4LexerMode, import metadata, logical paths, parser/lexer domains, modes, aliases, and both declared and effective dependency kinds as payload/provenance. Effective kinds make transitive full imports below a tokenVocab dependency explicitly lexer-only, while separate lexer-only/full provenance paths ensure an effective parser rule points to the full traversal that made it visible. The temporary G4ImportedRuleResolver delegates to that plan rather than maintaining another graph algorithm, and identifies an already supplied caller by its exact indexed payload so duplicate declared names do not override local-rule priority.

GrammarEmitter receives a mechanical effective projection from the shared plan rather than resolving imports itself. Generated definitions execute selected parser and lexer rules, fragments, modes, tokens, and channels; entry-owned root/options/actions and descriptive imports are retained. Project graph changes recompute emission, so removed imports do not leave stale declarations.

NuGet analyzer packaging strategy

The 2.0.0-rc.1 analyzer package places Utils.Parser.Generators.dll and its narrowly required Utils.Parser.Source, Utils.Parser.Diagnostics, and Utils.Parser.Antlr4.Common support assemblies together under analyzers/dotnet/cs. SuppressDependenciesWhenPacking prevents those compiler-host dependencies from becoming runtime dependencies in consuming applications. The package also supplies buildTransitive/omy.Utils.Parser.Generators.targets for generated-file attachment and compiler-visible options. The product-train acceptance suite builds before dotnet pack --no-build, inspects this exact layout, and compiles real package-only consumers with both EmitCompilerGeneratedFiles and UtilsParserAttachGeneratedFiles enabled and disabled.

Versioned API documentation

Package-only incremental acceptance

A real isolated consumer restored from the candidate feed builds direct/transitive imports and tokenVocab, executes the facade, modifies an imported grammar, removes an import, changes the vocabulary, and adds then removes a collision. Each rebuild executes the new effective composition and asserts removed rules do not remain. Separate failing builds verify UP0010, UP0011, and UP0016 from the packaged analyzer.