Eternet.TestPlanner.Build 1.1.125

Eternet.TestPlanner.Build

Eternet.TestPlanner.Build is an opt-in incremental source generator that emits a deterministic impact-index shard for each built C# project.

Enable it centrally with EternetTestPlannerImpactIndexEnabled=true. The generated carrier contains comment-only JSON below the target-framework-specific obj directory, so the payload is not compiled into the consumer assembly. The accompanying MSBuild target copies the carrier to EternetTestPlannerImpactIndexOutputPath and publishes an adjacent Eternet.TestPlanner.BuildReceipt.json through a separate MSBuild-task assembly. ETP validates the schema, compile inputs, configuration fingerprint, completeness, and (when present) the receipt-to-carrier SHA-256 binding before it accepts a shard for inspection. Materialized shards use a versioned layout and a SHA-256 identity derived from the project path, then isolate configuration, platform, target framework, and runtime identifier. This prevents projects that share an output root from overwriting one another. A consumer-defined CompilerGeneratedFilesOutputPath is preserved. Design-time and skip-compiler builds never materialize carriers. Missing, incomplete, or invalid shards must fall back to normal ETP analysis.

Advisory Razor origin evidence

For projects with @(RazorComponent) inputs, a post-compile task writes Eternet.TestPlanner.RazorOrigins.json beside the receipt. Schema etp.razor-origin.v1 binds the sidecar to the carrier and assembly SHA-256; the receipt records the sidecar path/SHA-256 and the assembly SHA-256. Each input has a workspace-relative path and SHA-256, a completeness flag with uncertainty reasons, matching generated-source paths and SHA-256 values, generated type names, and candidate metadata members (including compiler-created nested and lambda methods). A generated source matches an input only when its #pragma checksum names that exact source and its SHA-256 matches the current input bytes. The first slice recognizes generated classes with an explicit global::Microsoft.AspNetCore.Components.ComponentBase base. A checksum-bound _Imports.razor whose checksum-verified source contains only C# @using directives is recorded as a directive import when it generates no component type. Razor may emit boilerplate Execute IL for this source. An unchanged import does not make component type evidence incomplete. A changed import keeps its project-level impact bound because its directives can affect components in its directory. Custom base components remain incomplete. Linked inputs outside the workspace remain incomplete even when their generated source matches; their safe Link path is used for display. Input and generated-source locations use MSBuild FullPath metadata, so the build invocation directory does not change their identities. Syntax-derived type names retain generic arity for the component and any containing types. Generated files above 16 MiB or a total above 512 MiB make the affected sidecar incomplete before any source is parsed; sidecars above 8 MiB are skipped with a build warning. Type names come from C# syntax and members from the compiled assembly; generated filenames do not establish the binding.

The sidecar travels with a published snapshot. etp build-index reports its validation status, path, and complete/total input counts. An invalid sidecar does not change existing shard usability or authorize narrower selection. The receipt task writes the sidecar after any carrier reconciliation so its carrier hash binds the final carrier. The sidecar is an optional MSBuild output; absence does not force repeated materialization. MemberCandidates includes every compiled method under the generated class, so a method contributed by another partial declaration can be present. A future consumer must prove ownership and coverage against PDB/IL before using it for run2 narrowing.

Interface receiver proof is scoped to each owning method and contract. Escaping locals (including captured locals, ref aliases and ref/out arguments) invalidate only the receiver values they can affect. Unrelated lambdas and ref/out arguments do not invalidate a known receiver. Any unknown call site for the same contract still retains conservative dispatch for that entire owner/contract pair.

For straight-line bodies, the proof follows the last completed assignment at each call, including the value when a local was copied. Assignments after the call do not add targets. Multiple calls still union their proven targets. Branches, loops, exception handling, deferred bodies, embedded assignments and disposal retain the conservative union of assignments; this is not a full CFG analysis. Compiler errors, analysis limits and open virtual implementations never authorize narrowing. Legacy carriers without proof retain broad dispatch.

EternetTestPlannerBuildReceiptEnabled=false exists only to isolate local performance measurements; CI must leave the default (true) in force.

Test projects with no peer generated sources can opt into IL graph evidence with EternetTestPlannerAssemblyImpactForTestsEnabled=true. The switch takes effect only when IsTestProject=true and build receipts are enabled. Its default is off because writing the graph adds post-compile work and artifact size. The graph is listed as an incremental output only when it will be generated; turning the switch off removes a graph from a previous opt-in build and clears its receipt binding. Carrier reconciliation remains limited to projects with peer generated sources.

Incremental impact-index materialization

EternetTestPlannerMaterializeImpactIndex (after CoreCompile) declares MSBuild Inputs/Outputs, so an up-to-date project skips the receipt, peer manifest and assembly-impact rewrite entirely. EternetTestPlannerCollectImpactIndexInputs runs first and gathers the inputs: the generator carrier, peer generated sources, the intermediate .dll/.pdb, @(ReferencePathWithRefAssemblies), the prepare, assembly-reference and delta-input manifests, EAC diagnostic request/SARIF paths, the task assembly, and Eternet.TestPlanner.ImpactIndex.inputs.cache. That cache file is written with WriteOnlyWhenDifferent and fingerprints every scalar that shapes the outputs (project path, TFM, configuration/platform/RID, SDK version, mode, receipt and test graph switches, repository, diff base, SourceRevisionId, delta parent metadata, reference list, task assembly path), so a scalar change reruns the target without a recompile. Outputs are the copied carrier, the peer-sources manifest, the build receipt and Eternet.TestPlanner.AssemblyImpact.json.gz (or the delta fragment in delta mode); they are touched after a successful run so a subsequent no-op build is skipped even when a task wrote nothing new. When the generator emitted no carrier, or when EternetTestPlannerIncrementalImpactIndex=false, the target always runs (a never-written sentinel output forces it) and the existing missing-carrier error still fires. Delta mode rewrites its input manifest each build, so it always reruns; skipping applies to standard builds only. Set EternetTestPlannerIncrementalImpactIndex=false only to diagnose a suspected stale skip; outputs are byte-identical whether the target runs or is skipped.

EternetTestPlannerFinalizeBuildReceipt (after Build) refreshes the receipt on every build, including an up-to-date one, because Content, None and resource items can change without a recompile. Its expensive part is derived evidence: the IL graph, the carrier reconciled against it and the Razor origins. Within a build that compiled, the refresh takes the compile-phase graph from memory. Across builds it uses obj/Eternet.TestPlanner/receipt-evidence/<configuration>/<platform>/<tfm>/<rid>/receipt-evidence.json, a stamp recorded after the evidence was computed: the hash of every file it read (assembly, PDB, the PDB's source documents, generated sources, Razor inputs and the identity of each reference), the paths and switches that shape it, the task and contract assemblies, and the carrier, graph and Razor origins it left behind. When the stamp of the current files matches, the refresh keeps that evidence and only rebuilds the receipt from the current task parameters; any difference recomputes it. A stamp is not recorded when an input changed while the evidence was computed.

Canonical compiler prepare payload

Native affected-tests builds pass build/current-build-impact-prepare.mpack directly to the MSBuild tasks. The tasks read a bounded, typed projection of the LZ4-compressed MessagePack document; they do not expand or publish a JSON copy. MessagePack and its generated formatters belong to the task assembly, not the Roslyn generator. Only the small derived delta-input text manifest is exposed as a compiler additional file. The package carries MessagePack dependencies under buildTransitive, never under analyzers.

Upgrade the ETP tool and Build package together and start a fresh workflow run root. Older Build packages cannot read this binary prepare contract. An explicitly supplied JSON prepare remains readable, but ETP does not search for or generate a JSON fallback. Imported overlay prepares retain their exact bytes and format: their overlay binds the raw SHA-256, so transcoding after certification is invalid.

Changed-input delta mode

Set EternetTestPlannerImpactIndexMode=delta and provide EternetTestPlannerPrepareManifestPath to build only the changed-input evidence. The generator filters its incrementally cached source/declaration providers before collection and emits Eternet.TestPlanner.ImpactDelta.g.cs; it does not emit the complete ImpactIndex.g.cs carrier in this mode. The carrier and materialized fragment are proportional to the changed inputs.

The generated etp.build-impact-delta-carrier.v2 stores shared symbols and referenceSets, and each declaration/test names a referenceSet. Its reference storage budget counts distinct sets by content, plus declaration dispatch keys; the serialized payload retains its separate 48 MiB bound. A separate logical budget keeps the previous 500,000 declaration and 250,000 test relation limits across owners until their query and publication costs are measured. The MSBuild task validates the pool and expands it while writing the existing inline-reference fragment, so fragment fingerprints, the certified schema fingerprint, and etp.build-impact-delta.v2 semantics remain stable. Previously generated etp.build-impact-delta-carrier.v1 inputs remain readable. The task checks projected inline-reference bytes before expansion and the written fragment bytes before publication. Budget failures yield an incomplete fragment with an explicit reason.

The MSBuild task classifies current compile inputs as upserts, Git deletions as deletes, and changed files removed from Compile as tombstones. Set EternetTestPlannerImpactDeltaBaseRoot to the complete base snapshot when delete/tombstone operations are possible. Each inner build derives its own carrier below etp-layout-v3 from project + configuration + platform + TFM + RID + SDK version; callers do not enumerate carrier paths. EternetTestPlannerImpactDeltaBaseCarrierPath remains an explicit single-build override. The task reopens the selected complete base carrier, validates every identity field against the delta carrier, and derives the real base contribution fingerprint; absent, foreign, or ambiguous base evidence fails closed. An etp-layout-v2 snapshot is never considered a v3 base. CI must publish one complete full etp-layout-v3 seed before enabling PR deltas. An existing project build requires that compatible carrier; a project/build identity first introduced by the PR may emit only upserts with null base fingerprints. A new file in an existing project is represented by a missing contribution inside a present v3 carrier. Each inner build produces an isolated fragment with project, TFM, configuration, platform, RID, and SDK version identity, including multi-targeted and multi-RID projects. SDK/TFM/compiler-option changes alter the compiler fingerprint and invalidate incompatible checkpoint reuse.

etp.build-impact-delta-fragment.v1 carries operations only and explicitly has operation-fragment-not-certified-checkpoint authority. It becomes etp.build-impact-delta.v2 only through etp build-impact-delta compose after terminal plan/build/test receipts exist. Neither the generator nor MSBuild task claims that an in-progress compilation is a green checkpoint. For later commits in the same PR, EternetTestPlannerImpactDeltaParentManifestPath plus EternetTestPlannerImpactDeltaParentCheckpointFingerprint resolves the prior cumulative head contribution before consulting the full seed. This proves main → C1 add/modify → C2 delete across different workspaces without copying assemblies or walking a remote checkpoint chain.

The parent reader accepts an explicit .mpack/.msgpack checkpoint or JSON input. It projects only contribution identities: declaration/test graphs are skipped, and a single dictionary indexes operations by project, TFM, configuration, platform, RID, SDK and input path. It rejects duplicate identities, wrong parent fingerprints and corrupt binary input without retrying another format. Binary decompression and input size are bounded to 64 MiB with depth 64. This reader does not itself certify or publish a checkpoint.

The generator uses Roslyn incremental providers per declaration, test, source file, and build input. Test reference collection walks bound operations only and has hard per-test/project budgets; exceeding one marks the shard incomplete instead of spending unbounded compilation time or silently omitting evidence. The project-wide limits include total references and serialized payload size, so a pathological project cannot create an unbounded carrier.

The semantic edges include every attribute applied to a discovered test, constructor and named attribute arguments (including typed arrays), compiler- selected custom-await members, and same-project implementations reached through interface dispatch. Implementations in referenced projects remain protected by the recorded assembly MVID. The reader also reevaluates the exact recorded compiler/build configuration and rejects a shard if any relevant property has changed.

For System.Type values, the generator follows finite typeof alternatives in conditional and switch expressions and through local variables with one initializer and no later mutation. It retains the type fixture effects when those values escape or drive reflection. A set is limited to 32 types and 32 expression levels; an unknown alternative, mutable local, or exceeded analysis budget keeps the dynamic operation fallback. Nominal metadata queries do not activate the inspected type, but later reflection on an unresolved result still falls back.

Because Roslyn does not expose one generator's output to another in the same compilation, the post-compile target records peer-generated C# sources beside the carrier. Their presence makes the shard fail closed: another generator may have emitted a discoverable test that ETP could not observe.

This repository imports Eternet.TestPlanner.ImpactIndex.targets from every Directory.Build.targets boundary. Diff Affected Tests enables it for the single affected-project graph build, validates every emitted shard, and uploads the carriers plus the inspection manifest. Ordinary builds leave it disabled, so source/declaration analysis and generated-file I/O are not a fixed local or design-time tax.

Run the reproducible performance harness with:

dotnet run --project tools/Eternet.TestPlanner/benchmarks/Eternet.TestPlanner.Build.Benchmarks -c Release

See PERFORMANCE.md for the corpus, budgets, and local before/after measurements.

Method overload identities

Assembly impact retains the v2 name-only fields for older readers and adds optional SignatureId, SignatureReferences, and SignatureSourceOwnerId fields. New readers preserve representable parameter types (including arrays, pointers, and by-reference parameters). Generic substitutions, custom modifiers, function pointers, and unsupported IL signatures retain conservative method-family aliases. Old graphs also retain these bidirectional aliases, so mixing evidence generations cannot disconnect callers.

Source diff matching separates unchanged overload bodies only when their parameter identities can be proved syntactically from intrinsic C# type keywords and their containing types. Aliases, user type names, and generic signatures keep the broader comparison. Adding, removing, or changing a signature also keeps the broader comparison because an unchanged caller can bind to a different overload. This precision does not alter selection budgets or bypass incomplete compiler evidence.

Compact full carriers (v8)

Full compiler carriers use etp.build-impact.v8. A carrier stores a symbols string table and referenceSets arrays of zero-based symbol indexes. Every source declaration and test has one referenceSet index; identical bodies share that entry. Empty reference sets have an ordinary index. Consumers reject invalid indexes and owners that mix compact and inline reference fields. v7 carriers remain readable through the legacy inline-reference path.

A declaration records dispatchKeys: the interface/virtual slots it implements. The producer stores direct references and these source-owned relationships, without expanding every call into every implementation. After applying source deltas, the planner builds shared dispatch nodes per project/build identity and receiver-proof exclusion context. Unknown receivers retain all implementations; proof from one caller is never shared with a caller having different exclusions. Deleting an implementation removes its source-owned relationship before that projection is rebuilt. Source dispatch remains scoped to the compilation, as it was in v7. Cross-project behavior still relies on the existing IL graph and assembly-reference identity validation; v8 does not claim new DI evidence or merge unqualified symbols from unrelated assemblies.

The 48 MiB serialized-character cap remains. Reference budgets now charge physical unique direct-reference lists, with dispatch relations charged to the declaration budget. The reader bounds pool entries and the planner separately bounds its dispatch projection. Exceeding a bound retains conservative fallback. This format does not implement paged source contributions: a project whose compact evidence exceeds the cap still cannot produce a complete snapshot.

The receipt reconciles only unresolved owners into the shared reference pools; it does not create an expanded JSON graph. The shared decoder resolves each reference-set array once per load. Decoded data stays carrier/session scoped; workspace-dependent validation is not cached globally.

Changed-input delta payloads retain their bounded inline reference representation and include source-owned dispatchKeys. Contribution fingerprints bind those keys. The new dispatch schema has a distinct checkpoint fingerprint. Mixed seeds may retain unchanged complete v7 project/build carriers; only a project targeted by a new source-owned delta needs a complete v8 carrier reseed. Mixing old expanded and new direct source contributions inside that target project is rejected, because v7 did not retain all implementation ownership. An all-v7 seed keeps its legacy fingerprint. CLI and Build packages must be upgraded together.

When receipt reconciliation changes an owner, its pools are rebuilt from live owners only, with shared sets copied once and stale symbols/sets discarded. Pool, section-reference and serialized-payload budgets remain enforced. Overflow emits explicit incomplete evidence, including incomplete test inventory, rather than certifying a truncated graph.