WpfTestKit.Xunit 0.1.1

WpfTestKit

Reusable WPF testability infrastructure. Drops into any WPF app to give it a trace log, hidden UIA observability surface, DI-friendly service contracts, optional batch-job plumbing, and a FlaUI-based out-of-process click-test harness with auto screenshot + UIA-tree capture on failure.

The kit ships in two packages:

• WpfTestKit — diagnostics, services, batch, FlaUI helpers. Reference from your WPF app.
• WpfTestKit.Xunit — FlaUiTestFixture base class. Reference from your test project.

What your WPF app must do to use the kit

1. Put <diagnostics:DiagnosticsUiaSurface /> somewhere in MainWindow. Without it, FlaUI can't read the trace log out of the running process.
2. Put AutomationProperties.AutomationId on every element a test will touch.

That's the hard contract. Everything else is recommended but optional:

• DI is recommended but not required. AddWpfTestKit* extensions are sugar; new DiagnosticsService() works fine.
• CommunityToolkit.Mvvm is not a kit dependency. The kit uses its own hand-rolled INotifyPropertyChanged. Your VMs are free to use CTK, ReactiveUI, or nothing; the kit doesn't care and won't drag CTK into consumers.
• Your async commands don't have to use IBackgroundJobService. It's a testability tool for VM unit tests; FlaUI tests work regardless.

Consumer-side startup wiring is under 50 lines — see the example at the end of this file.

Where to go next

• Writing more than a handful of tests? Read the Testing patterns section below.
• Looking up a specific capability? Skip to the feature matrix at the bottom.

Testing patterns

The kit's primitives (full reference in the feature matrix below) compose into patterns that keep tests fast, deterministic, and maintainable. Pick the ones that fit.

Structuring tests

Arrange / Act / Assert. Launch → drive → assert. Dispose is automatic via FlaUiTestFixture.

[Fact]
public void IncrementButton_ClickedThrice_CounterReadsThree()
{
    // Arrange
    var app = MyAppTestApp.Launch();
    SetCurrentApp(app);

    // Act
    for (int i = 0; i < 3; i++) { app.InvokeById("IncrementButton"); }

    // Assert
    Assert.Equal("3", app.WaitForText("CounterText", "3", timeoutSeconds: 3));
    app.AssertNoDiagnosticsErrors(Output.WriteLine);
}

TestApp wrapper per app. Don't repeat Launch(exePath, args) in every test. One helper per app keeps scenario tests concise and lets you push common startup args (test-mode flags, stubbed services) into one place.

internal static class MyAppTestApp
{
    public static TestApp Launch(string? arguments = null)
        => TestApp.Launch(TestApp.LocateExeRelativeToSolution("MyApp"), arguments);

    // Common test-mode variants — adopt as needs surface:
    public static TestApp LaunchWithStubbedFileDialog(string? pathToOpen)
        => Launch($"--test-file-to-open={pathToOpen ?? "CANCEL"}");
}

Screen-object helpers for repeated flows. When a five-line login/save/load sequence appears in eight tests, extract a static helper that takes a TestApp. Keep it thin — it's a named composition, not an abstraction layer.

internal static class LoginFlow
{
    public static void SignInAs(TestApp app, string user, string pw)
    {
        app.TypeInto("UsernameInput", user);
        app.TypeInto("PasswordInput", pw);
        app.InvokeById("SignInButton");
        app.WaitForText("WelcomeLabel", $"Welcome, {user}", timeoutSeconds: 5);
    }
}

Extending FlaUiTestFixture. If you want custom capture artifacts, startup checks, or per-suite diagnostics, override DisposeCore — don't subclass Dispose directly. The base already handles app disposal + artifact capture policy. See ArtifactCapturePolicy for Always / OnFailure / Never.

Element access

Prefer InvokeById / TypeInto over raw FindById. The one-shot helpers auto-retry until the element exists (default 3s). FindById returns AutomationElement? immediately — you lose auto-retry and have to handle null yourself. Use FindById only when you genuinely need a "is it there right now?" predicate.

Use Locator when touching the same element more than once. Locator re-queries UIA on every action, so it survives layout rebuilds (tab switches, collection changes, anything that invalidates the previous element reference).

var save = app.Locator("SaveButton");
save.Click();           // first interaction: fresh query
// ...UI may rebuild...
save.Click();           // re-queries; safe even after layout changes
var text = save.Read(); // same handle, different action

Use LocatorIn for modals and secondary windows. Same stale-proof behavior, but rooted at a given AutomationElement instead of MainWindow. Essential when InvokeAndWaitForModal returns a modal you want to drive with the same ergonomic API.

var modal = app.InvokeAndWaitForModal("DeleteButton", "Confirm", timeoutMs: 3000);
app.LocatorIn(modal!, "DialogOkButton").Click();

Waiting and synchronization

Never Thread.Sleep to wait for state. The kit ships four waiters; one of them fits every real case.

Thread.Sleep is only acceptable for documented UIA-level races (e.g. the 150ms settle after SelectTabItem, already in the kit). Your test code shouldn't need it.

Don't add manual retry around kit calls. Auto-retry is already there. InvokeById("Foo") retries until DefaultActionTimeoutMs (3s). If you need a longer budget, pass timeoutMs: explicitly — don't wrap in a loop.

Use Retry(action, maxAttempts, delayMs) only for genuinely flaky scenarios that auto-retry can't fix (e.g. a background service that only settles after an external event). Document why — future-you will thank you.

Async commands and background work

Trigger the command, then WaitForText the observable result. Don't poll internal VM state from a FlaUI test — you can't see it anyway, and the label is the user-visible contract.

app.InvokeById("StartSlowJobButton");
var status = app.WaitForText("JobStatusLabel", "Completed", timeoutSeconds: 30);
Assert.Equal("Completed", status);

For the VM-level unit tests you should write separately from FlaUI tests, swap the production IBackgroundJobService for InlineBackgroundJobService (runs synchronously) or GatedBackgroundJobService (blocks until you Release()) so you can observe mid-flight state without races.

Modals

Expected modal → InvokeAndWaitForModal. Fires the trigger on a background task so the modal's nested message loop can't block the test thread. Returns the modal window; drive it with LocatorIn or InvokeButtonByName / InvokeButtonById.

Unexpected modals → StartModalWatchdog. Stands up a background poller that auto-dismisses pop-ups using a safe default button order (Cancel/No/Close/OK/Yes). Every test that doesn't expect a modal should have one.

using var watchdog = app.StartModalWatchdog();
// ...exercise flow that shouldn't open anything...
Assert.Equal(0, watchdog.DismissedCount);   // hard failure if it did

Combine both patterns: one test expects "Confirm deletion", every other test uses the watchdog as a safety net.

Assert on the modal's content, not just that it appeared.

app.AssertContainsText(modal!, "Proceed with delete?");
Assert.True(app.HasButton(modal!, "Yes"));
Assert.True(app.HasButton(modal!, "No"));

Keyboard

Pick the right layer for what you're actually testing:

Reach for SendKeys / PressShortcut only when the keystroke handler is what's under test. For state setup, TypeInto is faster and doesn't require window focus.

Assertions

Assert against durable observable state. In order of preference:

1. UIA property — ReadText, IsToggled, GetSelectedComboBoxText, CheckReachability. These are what the user experiences.
2. Diagnostics — AssertNoDiagnosticsErrors(Output.WriteLine) at the end of every test. Catches silent exceptions, data-binding errors, and anything routed through IDiagnosticsService.
3. Visual regression — AssertScreenshotMatches(baselineName, baselineDir, diffTolerancePct, perPixelTolerance) for layout-sensitive screens. First run establishes the baseline; subsequent runs diff. Adjust perPixelTolerance for ClearType anti-aliasing drift.
4. Contrast — AssertContrastRatio(id, minimumRatio: 4.5) for WCAG AA on text elements that should be readable.

Don't assert on things the user can't see (internal VM state, timing, exact pixel counts). If your test breaks on an innocuous refactor, the assertion was too narrow.

Reachability — guarding before you interact

For tests that care about layout (resizing, tab-content, scroll containers), assert reachability before the action:

app.ResizeMainWindow(400, 300);
app.AssertReachable("SaveButton");   // throws with reason if clipped, covered, off-screen
app.InvokeById("SaveButton");

Or for a whole-screen hygiene check:

var audit = app.AuditAutomationIds();
Assert.True(audit.IsClean,
    $"{audit.MissingOnInteractiveElements.Count} missing, {audit.DuplicateIds.Count} duplicate");

Reachability distinguishes NotFound / HitTestBlocked / ScrolledOutOfView / OutsideWindow / PartiallyClipped / ZeroSized / OffscreenViaUia with a reason string — failure messages tell you why it wasn't clickable, not just that it wasn't.

Diagnostics and triage

FlaUiTestFixture captures two artifacts on every failure by default — screenshot + UIA tree dump to %TEMP%\WpfTestKit\. Paths are echoed to test output.

Triage order when a test fails:

1. Read the error message. IdSuggester already suffixes lookup failures with "Did you mean: 'XxxButton'?" — most typos resolve here.
2. Open the screenshot. Is the app in the state you expected? Wrong tab, wrong modal, crashed?
3. Open the UIA tree dump. Does the element you named actually exist? With that AutomationId? Under the expected parent?
4. Re-run with TraceRecorder for step-level timing. Attach via app.Trace = new TraceRecorder(); dump HTML at end of test. Most useful for "it works sometimes" bugs.
5. Mid-test app.Pause("why") to freeze the running app and inspect interactively. No-op when WPFTESTKIT_HEADLESS=1 — safe to leave committed.

Flakiness mitigation

The kit's auto-retry eliminates most manual flakiness control. The remaining levers:

• Pick semantic waits over timeouts. WaitForText(id, "Done", seconds: 30) is far more robust than Thread.Sleep(30_000) — it returns as soon as state changes.
• Isolate one app per test. The default FlaUiTestFixture gives you that. Sharing an app across tests to save launch time is almost never worth the cross-test contamination.
• Use AssertNoDiagnosticsErrors in every test. Catches errors that would otherwise present as unrelated downstream flakes.
• For known races, WaitFor(() => predicate) not Thread.Sleep. Returns on first success.
• Screenshot diffing with tolerance. diffTolerancePct: 0.5, perPixelTolerance: 10 handles ClearType subpixel drift; tune higher if your test machine uses a different DPI than the baseline.
• If a specific test is still flaky despite the above, wrap it with app.Retry(action, maxAttempts: 3) and leave a comment explaining the external cause. Don't retry as a blanket policy.

Recording new tests

Covered fully in User guide §6. Summary: when starting from scratch on an app, launch the recorder, drive the flow manually, copy generated [Fact] code, add assertions where the // TODO lands.

Anti-patterns

Things that look reasonable but cause flakes, false passes, or maintenance debt:

• Thread.Sleep instead of a waiter. Auto-retry and WaitForText / WaitFor cover every real case. If you're tempted, re-read §Waiting.
• Hardcoded paths to MyApp.exe. Use TestApp.LocateExeRelativeToSolution — tests run on any dev machine without path edits.
• Asserting on timing (elapsed < 100ms). Test machines vary by 10×. Assert on behavior; measure timing separately if you care about performance.
• Sharing a TestApp instance across tests. Cross-test state contamination is the #1 source of "passes alone, fails in suite" flakes. Default is one app per test — stick with it.
• Skipping AssertNoDiagnosticsErrors. Binding errors, unhandled exceptions in command handlers, and async void throws all flow through it. Opting out means those bugs become next month's flake.
• Using FindById + null check where InvokeById would do. You lose auto-retry and write more code. Only meaningful when you specifically want "is this visible right now?".
• Asserting on AutomationElement.Name instead of AutomationId. Name is localized and changes with the user's OS language; AutomationId doesn't.
• Pausing in CI. Set WPFTESTKIT_HEADLESS=1 — Pause() becomes a no-op. Never leave a Pause() call blocking CI.
• Relying on focus for assertions. Focus state is fragile (foreground stealing, alt-tab). Use UIA property reads (IsToggled, ReadText) which work regardless of focus.
• Adding a screenshot assertion to a fast-changing UI. Visual regression is for stable layouts (dialog chrome, splash screens). Scenery that shifts every sprint will just rot the baseline.
• Testing implementation details (command names, VM property order). Test what the user sees through UIA. If you need VM-level coverage, write separate unit tests against the VM directly.

Feature × kit capability — extensive matrix

Legend: ✅ shipped and exercised by an end-to-end test · 🔧 shipped in kit, unit-tested only · 🚧 planned (design sketched) · 💭 idea (wanted; needs design)

Basic interaction

Tabs, dialogs, and secondary windows

Async commands & concurrency

Failure observability

Reachability & layout (NEW — the point of WpfTestKit that nobody else solves)

OS file dialogs (preferred: stub via DI; fallback: watchdog)

Keyboard & input

Data editing

Evidence capture on failure

Debugging aids (mid-test inspection)

Integration wiring

Resource hygiene & teardown

Audits & hardening (no consumer trigger; run once per suite)

Consumer-side wiring (App.xaml.cs)

public partial class App : Application
{
    private IServiceProvider? _services;

    protected override void OnStartup(StartupEventArgs e)
    {
        base.OnStartup(e);

        var services = new ServiceCollection();
        services.AddWpfTestKitDiagnostics();
        services.AddWpfTestKitFileDialogs();
        services.AddSingleton<MainViewModel>();
        _services = services.BuildServiceProvider();

        var diagnostics = _services.GetRequiredService<IDiagnosticsService>();
        DiagnosticsExceptionSinks.AttachTo(this, diagnostics);
        DiagnosticsUiaSurface.DefaultDiagnostics = diagnostics;

        new MainWindow(_services.GetRequiredService<MainViewModel>()).Show();
    }
}

And in MainWindow.xaml:

<Window xmlns:diagnostics="clr-namespace:WpfTestKit.Diagnostics;assembly=WpfTestKit" ...>
    <Grid>
        
        <diagnostics:DiagnosticsUiaSurface />
    </Grid>
</Window>

That's enough for FlaUI tests to find the diagnostics surface and assert on errors.