Runic.Application.Testing 0.8.0-preview.2

Runic.Application.Testing

Test a generated Runic Window contract and its routed Views without a browser or native window. RunicWindowTestHost<TViewModel> owns an in-memory transport, a real WindowContentSession, and the generated root Bridge attachment, so tests run the real ViewModels and generated .NET handlers. The caller owns the ViewModel and its dependency injection scope.

Testing

Build the application's own services, create a scope for the window, and create the host from it, with a test clock in place of the system clock:

var clock = new FakeTimeProvider();   // Microsoft.Extensions.Time.Testing
await using var services = new ServiceCollection()
    .AddSingleton<TimeProvider>(clock)
    .AddNotes()          // the application's registrations, including AddRunicViews()
    .BuildServiceProvider();
await using var scope = services.CreateAsyncScope();
using var host = RunicWindowTestHost.Create<ShellViewModel>(scope.ServiceProvider,
    new RunicWindowTestHostOptions { TimeProvider = clock });

RunicWindowTestHost.Create resolves what the Desktop and CS-WebUI windows resolve from the window's scope: the root ViewModel, its generated Bridge attachment (AddRunicViews() registers it), and, when registered, the IRunicViewLocator, the IRunicModelContext (for example from AddRunicReactiveModelContext() or AddRunicNavigation()) and the ILoggerFactory. A locator, context or logger factory set in the options is used instead. It names the missing registration when the ViewModel or its Bridge is not registered. The scope and its ViewModels stay yours; dispose the host before the scope.

The constructors take the ViewModel and the attachment directly, for a window that is not composed with dependency injection:

using var host = new RunicWindowTestHost<ShellViewModel>(shell,
    (transport, content, vm) => new ShellBridge(transport, vm, content: content),
    new RunicWindowTestHostOptions { ViewLocator = locator, TimeProvider = clock });

Then drive the Window by ViewModel member, as the generated TypeScript client does:

var sidebar = host.Root.View<SidebarViewModel>(vm => vm.Sidebar);
(await sidebar.ExecuteAsync(vm => vm.OpenNotesCommand)).EnsureOk();
var editor = host.Root.View<DocumentViewModel>(vm => vm.Main)
    .View<EditorViewModel>(vm => vm.CurrentPane);
editor.Set(vm => vm.Title, "Groceries").EnsureOk();

var save = editor.Start(vm => vm.SaveCommand);
Assert.True(editor.Snapshot().IsExecuting(vm => vm.SaveCommand));
clock.Advance(TimeSpan.FromMilliseconds(250));   // storage waits on the injected clock
Assert.Equal("succeeded", (await save.WaitAsync()).Kind);
Assert.Equal("Saved Groceries", editor.Snapshot().Read(vm => vm.SavedMessage));

dotnet new runic-app --tests creates an application with a TUnit test project that does this for its Window; the CommunityToolkit Notes example is a larger xUnit project.

Skip the frontend build

A test project that references the application project builds the application, and by default that build installs and builds the frontend too. The host needs only the .NET assembly, so pass RunicApplicationFrontendBuildEnabled=false to the reference, as the template's test project does:

<ProjectReference Include="../MyApp/MyApp.csproj" AdditionalProperties="RunicApplicationFrontendBuildEnabled=false" />

The generated TypeScript clients are still written. In a solution that also builds the application itself, MSBuild then builds the application project twice, once with each setting; build the test project on its own, or leave the property out and let the frontend build's up-to-date check skip unchanged frontends.

Deterministic time and ids

With RunicWindowTestHostOptions, the window uses a FakeTimeProvider from Microsoft.Extensions.TimeProvider.Testing, which this package depends on, unless the options supply another TimeProvider. Its time passes only through Advance or SetUtcNow, which fire due timers, CancellationTokenSource timeouts and Task.Delay(TimeSpan, TimeProvider) in order; a timer due now fires at once. The window uses it for interaction deadlines and BeginCloseAsync timeouts; register the same instance in the application's services for its own time. host.Time returns it.

Work that waits on the clock does not finish by itself:

• ExecuteAsync of an awaited command that waits on the clock completes only after the test advances it. Start the call, advance the clock, then await the call:

  var save = editor.ExecuteAsync(vm => vm.SaveCommand);
  clock.Advance(TimeSpan.FromMilliseconds(250));
  (await save).EnsureOk();
  

  Start returns at admission, so an operation can be inspected while it runs.

• Only code that takes the TimeProvider follows it. Task.Delay(TimeSpan), DateTime.UtcNow and ReactiveUI's schedulers (Throttle, Timer, RxApp.MainThreadScheduler) keep using real time. Give such code the clock, for example through a scheduler built on it, or test it with ReactiveUI's TestScheduler.

Content ids are 1, 2, ... in presentation order, so routes are content1, content2, ... and snapshots are stable. Set CreateId for another sequence. The drivers issue mount tokens test:mount1, ... and operation request ids request1, ....

The constructor without options keeps the system clock and random ids.

File watchers

RunicPathWatchTester drives the RunicPathWatchers started with its Options (see Watching files): their debounce and polling run on its FakeTimeProvider, Time. Give the options to the ViewModel that starts the watcher:

var watching = new RunicPathWatchTester();
using var overview = new OverviewViewModel(context, watching.Options);

File.WriteAllText(Path.Combine(checkout, "flake.lock"), lockText);
var change = await watching.NextChangeAsync();     // after the callback ran in the model context

// A path the test cannot change: report it as if the platform had.
await watching.TriggerAsync("/run/current-system");

• NextChangeAsync returns the next change not returned yet, once its callback has run. While none is reported, it advances Time one debounce interval at a time, so events that arrive from a real file system are delivered. It throws TimeoutException after 10 seconds of real time.
• TriggerAsync(path) marks the targets with that path changed in every active watcher, whether or not they changed, and returns NextChangeAsync. It throws InvalidOperationException when no active watcher has such a target.
• DrainAsync advances Time for 300 ms of real time and returns what was reported meanwhile, to assert that a burst produced one change or nothing more.
• ActiveWatchers counts the watchers that are started and not disposed, to check that a ViewModel stops its watcher.

Drivers

host.Root drives the root ViewModel. View<T>(vm => vm.Content) and Views<T>(vm => vm.Items) follow the content references in the current snapshot and check that they present T; host.View<T>(reference) drives a reference you already have. A RunicViewDriver<T> has:

• Snapshot(): a RunicViewState<T> with Read(vm => vm.Title) for scalars (a string, Boolean, Int32, Int64, double, decimal, GUID or enum), Get for the raw JSON of other values, Reference, References, Keys of a [RunicCollection], CanExecute, IsExecuting and FieldVersion. Read of a record, DTO or collection throws NotSupportedException naming the property; use Get(vm => vm.Item).GetProperty("wireField") instead.
• Set(vm => vm.Title, value), encoded as the generated client encodes it: an int, bool or non-nullable string as is, and other scalars as wire JSON (Int64 and decimal as strings, enums by wire name, null for nullable values). Use SetJson with the wire JSON of a DTO, list or union.
• ExecuteAsync(vm => vm.SaveCommand, argumentJson), which waits for an asynchronous command, and CanExecute(command, argumentJson) for commands with an argument. A reply has Ok, ErrorKind, ErrorMessage and State; EnsureOk() throws for a failed call. A command's declared failure ([RunicFailure]) replies with ErrorKind domain-failed and the encoded failure as Failure, such as {"$case":"titleRequired"}.
• Start(command, argumentJson, requestId) for a recoverable operation, with Status(), WaitAsync() and Cancel(). A status has Kind, Result, ErrorMessage and, for a domain-failed operation, Failure.
• Write(vm => vm.Title, value, expectedVersion) for a checked write. It uses the current value and version as the baseline unless expectedVersion is given, so a stale version tests the conflict receipt.
• Mount() and Unmount(token) acknowledge a browser presentation.
• TakePublications() and NextPublicationAsync() return this route's pushed frames in order: State, Delta with its RunicCollectionChanges, or Failure.

Navigation

A NavigationRegion<TContent> slot is an ordinary content slot, so the same host and drivers apply. Create the navigator with the window's model context and pass that context as RunicWindowTestHostOptions.ModelContext; a generated Bridge rejects a session whose context is not the navigator's.

using Runic.Navigation; // RunicNavigator and RunicModelContext
await using var context = new RunicModelContext();
await using var navigator = new RunicNavigator(new() { ModelContext = context, TimeProvider = clock });
var shell = new ShellViewModel(navigator);
using var host = new RunicWindowTestHost<ShellViewModel>(shell, attach,
    new RunicWindowTestHostOptions { ModelContext = context, TimeProvider = clock });

// A scripted guard: the test page's CanDepartAsync returns decision.Task, which
// holds the transition in Guarding until the test decides.
var decision = new TaskCompletionSource<bool>();
editor.Departure = decision.Task;
var back = shell.Main.BackAsync();
decision.SetResult(true);
await back;
await navigator.WhenIdleAsync();
var home = host.Root.View<HomeViewModel>(vm => vm.Main); // the resumed entry

• RunicNavigator.WhenIdleAsync() waits until no transition or retirement is running.
• navigator.UnretiredEntryCount counts entries that have not finished retiring (zero after DisposeAsync), and host.Content.RetainedContentModelCount(), an extension on the window's WindowContentSession, counts the content models whose model-context leases the window holds.
• Pass a FakeTimeProvider as RunicNavigatorOptions.TimeProvider to drive the close timeout and the overrun warning (event 1067) without waiting.

With AddRunicNavigation(), create the window scope with CreateAsyncScope(), because the navigator is only IAsyncDisposable. RunicWindowTestHost.Create gives the window the scope's IRunicModelContext, which the navigator shares:

await using var scope = services.CreateAsyncScope();
using var host = RunicWindowTestHost.Create<ShellViewModel>(scope.ServiceProvider,
    new RunicWindowTestHostOptions { TimeProvider = clock });

Write navigation tests so they are deterministic:

• Hold a transition in a phase with a TaskCompletionSource that a guard, InitializeAsync or ResumeAsync awaits. Signal a second one from the hook when the test must know that the hook started. Then the test decides when the hook continues, instead of racing it.
• Don't await a request whose guard waits for the test. Start it, wait for the observable effect, answer, then await it. For example, start the Back, wait until the dialog region's Current is set, execute the dialog's command through its driver, and then await the Back. A command driver's Start returns at admission, which suits a command that navigates.
• Wait for region changes with PropertyChanged, not with delays. Every change is raised inside a model turn.
• await navigator.WhenIdleAsync() before you assert on retirement: departing entries retire after the commit turn, and owned content is disposed then.
• Assert lifetimes through the counters. navigator.UnretiredEntryCount falls as entries retire, and is zero after DisposeAsync. RetainedContentModelCount() falls when owned content's leases are released. Assert Retired states and disposal counts of your own test content too.
• Make overruns and timeouts happen by advancing the FakeTimeProvider: 5 seconds for event 1067, and CloseTimeout for disposal.
• To provoke a race, drive overlapping requests from a seeded Random, and assert invariants that don't depend on timing. For example, every request ends, an entry retires at most once, and nothing is left unretired after DisposeAsync.

The Notes tests cover a guard that asks in a dialog region with PushForResult, resuming a retained entry, retirement of a parent's child regions, racing navigation and closing the window while the dialog asks.

Interactions

A test answers a ReactiveUI Interaction, such as the question of a ReactiveLeaveConfirmation.WithInteraction guard, with a .NET handler on the ViewModel. ReactiveUI asks the latest handler first, so a handler the test registers answers before a fallback registered in the ViewModel's constructor; dispose it to restore the fallback. From the ReactiveUI WPF navigation tests:

var answers = new Queue<bool>([false, true]);
var prompts = new List<string>();
using var handler = detail.ConfirmDiscard.RegisterHandler(context =>
{
    prompts.Add(context.Input);
    context.SetOutput(answers.Dequeue());
});

This is the supported test path for a question that a web View answers in the browser. The host does not run the generated TypeScript client, and a driver's Mount() acknowledges a presentation without registering any interaction handler, so the question never waits for a browser: it goes to the .NET handlers. The test still covers everything the ViewModel decides: the guard, the prompt's input, and what each answer keeps or discards. With no handler that answers, the departure fails closed with ReactiveUI's UnhandledInteractionException, as in the application.

The browser's side is tested with the frontend: the generated typed mock's interactions.<name>.request(input) resolves with the answer of the handler the View registered with interactions.<name>.handle(...); see the Testing guide. Which View is asked for which departure, and the fallback when none is mounted, are covered by the SDK's own tests and described in Leave confirmation in a web View.

CommunityToolkit.Mvvm has no interaction type: a Toolkit guard asks in a dialog region, and a test answers it by executing the dialog's command through its driver, as in Navigation.

Collections and deltas

Track() follows a route from a snapshot through its frames, as the generated client does: newer full states replace the state, deltas must continue the tracked revision and name the row keys they change, and failure notices are collected. A frame the client could not apply throws, naming the problem: a lost or reordered frame, an index outside the rows, a row key that does not match, an item count that does not match the keys, or an empty or duplicate key.

var tracker = host.Root.View<HomeViewModel>(vm => vm.Main).Track();
library.Record("Groceries", "Milk, eggs");
var state = await tracker.WaitUntilAsync(state => state.Keys(vm => vm.RecentNotes)[0] == "Groceries");
Assert.Equal(0, tracker.FullStates);                       // only keyed changes were sent
Assert.Contains(tracker.Changes, change => change.Kind == "move");
tracker.Verify();                                          // the frames match a fresh snapshot

Publications are delivered off the calling thread, so waits use real time, RunicWindowTestHostOptions.PublicationTimeout (five seconds by default). A route's frames go to its tracker or to TakePublications, not both.

Lower level

host.Transport is the in-memory transport: Call(...) and CallAsync(...) invoke any route with ViewTestArguments, and DrainPublications() returns raw publications (drivers then miss them). Mount, Unmount and Content.ReleaseConnection test View lifetimes and client disconnects. Await BeginCloseAsync before disposing a host that has accepted background operations; with a fake clock, its timeout passes only when the clock advances.

The host does not instantiate the application's native Window wrapper or run the generated TypeScript client or a rendering framework. Test the frontend with the generated typed mocks of @runic-artifex/views/mock; see its Testing guide. To drive the whole application in a browser, use RunicServeOnlyApplication (below); to run script in the embedded Runic Desktop View from an in-process test, use DesktopBridgeWindow<TViewModel>.Surface.ExecuteJavaScriptAsync. See the Runic.Application.Views.Desktop automation section.

The whole application in a browser

RunicWindowTestHost runs the ViewModels without a page. To test the real page against the real ViewModels, start the built application in serve-only mode with RunicServeOnlyApplication and point a browser at its URL. It works with an application on Runic Desktop or CS-WebUI and with any browser driver. With Playwright for .NET and TUnit:

[Test]
public async Task IncrementsTheCount()
{
    // The application, built next to the tests, for example through a ProjectReference.
    var start = RunicServeOnlyApplication.CreateStartInfo(
        Path.Combine(AppContext.BaseDirectory, "MyApp.dll"));
    await using var app = await RunicServeOnlyApplication.StartAsync(start);

    using var playwright = await Playwright.CreateAsync();
    await using var browser = await playwright.Chromium.LaunchAsync();
    var page = await browser.NewPageAsync();
    await page.GotoAsync(app.Url.ToString());
    await page.GetByRole(AriaRole.Button, new() { Name = "Increment" }).ClickAsync();
    await Assertions.Expect(page.GetByTestId("count")).ToHaveTextAsync("1");

    var children = app.GetDescendantProcesses();   // while the application runs
    await browser.CloseAsync();
    await Assert.That(await app.StopAsync()).IsEqualTo(0);
    await Assert.That(children.All(child => child.HasExited)).IsTrue();
}

StartAsync sets RUNIC_APPLICATION_SERVE_ONLY=1, removes DISPLAY and WAYLAND_DISPLAY, so no window can open, and waits up to 30 seconds (its timeout) for the first RUNIC_APPLICATION_URL= line. It reports an application that exits or stays silent with its standard output and standard error, which StandardOutput and StandardError also return. Add arguments or environment variables to the ProcessStartInfo before starting it; CreateStartInfo runs a .dll with dotnet. StartAsync uses the file name, arguments, working directory and environment and leaves the ProcessStartInfo unchanged, so it can start the application again.

StopAsync writes a line to standard input, which ends serving on every platform, and returns the exit code. When the application has not exited within its timeout (30 seconds), its process tree is killed and TimeoutException is thrown. The line is written once: another StopAsync, for example after a cancelled one, waits for the same exit with its own timeout, and after a kill it throws InvalidOperationException. Disposing stops the application the same way but kills a hanging one without an exception, so call StopAsync when a test should fail on a shutdown that hangs. To check that the application ends the processes it started, such as a long-running tool it cancels on shutdown, list them with GetDescendantProcesses while it runs and check HasExited after StopAsync. A process whose parent has exited is no longer listed as a descendant.

A test process that exits without disposing the application still ends it on a best-effort basis: on Windows the application runs in a job object that ends with the test process, even when the test process is killed, and elsewhere the application is killed when the test process exits normally. A test process that is killed itself on Linux or macOS, for example with SIGKILL or by a crash, can leave the application running.

The page connects to the real Bridge, and the serve-only surface admits one browser. A test in TypeScript can follow the same protocol, described in the Runic.Application.Views.Desktop automation section: start the application with RUNIC_APPLICATION_SERVE_ONLY=1, read the RUNIC_APPLICATION_URL= line, and write a line to its standard input to stop it.