YAAQS.Xunit 0.9.1

YAAQS — Yet Another Automation Quick Start

A modern, MTP-first, .NET 10, unit-test-agnostic selenium/playwright framework loosely based on legacy frameworks like Magenic MAQS (until later acquired multiple times until the framework was essentially abandoned).

It keeps the things teams relied on MAQS for, but is redesigned so the automation core is not coupled to a test runner — it can be used from a console app, a worker service, or any host, as well as from tests. It supports both Selenium and Playwright, each as an independent package, on your choice of TUnit, NUnit, MSTest or xUnit.

No legacy coupling in the core. Unlike MAQS (which leaks Microsoft.VisualStudio.TestTools.UnitTesting and NUnit into even its Utilities/Base assemblies), YAAQS.Core, YAAQS.Selenium and YAAQS.Playwright reference no test framework at all. Test-framework support is opt-in via thin adapter packages. Selenium and Playwright are fully independent — a Selenium consumer never transitively references Playwright, and vice-versa. All adapters require Microsoft.Testing.Platform (MTP), not the legacy VSTest runner.

Table of contents

• Architecture
• Quick start
  • TUnit · NUnit · MSTest · xUnit (v3)
• Configuration
• Azure DevOps test attachments
• Migration notes from MAQS
• Building and testing
• Project layout

Architecture

A runner- and technology-agnostic core, one library per technology, a shared framework-binding layer, and a thin per-technology leaf adapter. You take only the packages you need:

  Selenium test consumer                              Playwright test consumer
        │                                                       │
  YAAQS.Selenium.<Fw>   (SeleniumTest base)        YAAQS.Playwright.<Fw>  (PlaywrightTest base)
   refs YAAQS.<Fw> + YAAQS.Selenium                 refs YAAQS.<Fw> + YAAQS.Playwright
          \                                                    /
           \                ┌──────────────────────┐          /
            └──────────────▶│ YAAQS.<Fw>           │◀────────┘     <Fw> = TUnit | NUnit | MSTest | Xunit
                            │  AutomationTest +     │  the shared, technology-free framework binding
                            │  <Fw>ResultSink       │  (hooks + IResultSink); refs YAAQS.Core + the
                            │  refs Core + framework│   test framework ONLY — no Selenium, no Playwright
                            └───────────┬───────────┘
   ┌────────────────────────┐          │          ┌────────────────────────┐
   │ YAAQS.Selenium         │          │          │ YAAQS.Playwright        │
   │  WebDriverFactory,     │          │          │  PlaywrightDriverFactory,│
   │  SeleniumObject,       │          │          │  PlaywrightObject,       │
   │  SeleniumLifecycle …   │          │          │  PlaywrightLifecycle …   │   ← use either from a console app
   │  (NO test framework)   │          │          │  (NO test framework)     │
   └───────────┬────────────┘          │          └────────────┬───────────┘
               └─────────────────┬─────┴───────────────────────┘
                       ┌─────────▼──────────┐
                       │ YAAQS.Core         │  Logging, Configuration, Waits, AutomationObject, DriverManagers,
                       │  (NO technology,   │  SoftAssert, Performance, AutomationLifecycle (sync + async),
                       │   NO test fwk)     │  IResultSink — the single seam that decouples automation from a runner
                       └────────────────────┘

• The single seam that decouples automation from a test framework is IResultSink (in Core): it exposes the only two things automation code needs from a runner — the pass/fail outcome and "attach this file to the result". Each YAAQS.<Fw> ships one IResultSink (TUnitResultSink, …) plus the generic AutomationTest base wired to that framework's lifecycle hooks.
• Selenium ↔ Playwright separation is enforced at the project-reference level. The shared YAAQS.<Fw> references neither technology, so the per-technology leaf packages (YAAQS.Selenium.<Fw> / YAAQS.Playwright.<Fw>) are the only place a technology enters — and a consumer of one never drags in the other.
• The framework-agnostic teardown orchestration lives once in AutomationLifecycle (Core); the shared browser + capture-on-failure logic lives once per technology in SeleniumLifecycle / PlaywrightLifecycle. Because Playwright is async-only, AutomationLifecycle exposes an async path (SetupAsync/TeardownAsync) used by every adapter.

Namespaces are stable across frameworks so your test code is identical no matter which you pick: YAAQS.Selenium.Testing (SeleniumTest), YAAQS.Playwright.Testing (PlaywrightTest), and YAAQS.Testing (the generic AutomationTest, for non-browser automation). You only swap the package reference and your test attribute ([Test] / [TestMethod] / [Fact]).

Quick start

Every adapter runs on Microsoft.Testing.Platform (MTP): the test project is an executable (OutputType=Exe) and must not reference Microsoft.NET.Test.Sdk. Pick your test framework below — each section shows the same task two ways:

• With the test framework — derive from the adapter base (SeleniumTest / PlaywrightTest); per-test browser setup, logging, capture-on-failure and result attachment all happen automatically.
• Without a test framework — construct the automation object (SeleniumObject / PlaywrightObject) directly from any host (console, worker, CI script). These standalone snippets are identical regardless of which test-framework package you reference — YAAQS.Selenium and YAAQS.Playwright have zero test-framework dependency — so they're shown in full under TUnit and referenced from the other sections.

Across frameworks, only the test attribute ([Test] · [TestMethod] · [Fact]), the [TestClass] MSTest requires, and the assertion API differ. Selenium Manager resolves Selenium drivers automatically; Playwright needs a one-time browser install.

TUnit

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="TUnit" Version="1.56.25" />
  </ItemGroup>
  <ItemGroup>
    
    <ProjectReference Include="..\..\src\YAAQS.Selenium.TUnit\YAAQS.Selenium.TUnit.csproj" />
  </ItemGroup>
</Project>

Microsoft.Testing.Extensions.TrxReport flows in transitively from the adapter, so --report-trx yields a TRX with attachments.

Selenium + TUnit

With a test framework:

using YAAQS.Selenium.Testing;

public class LoginTests : SeleniumTest
{
    [Test]
    public async Task CanReachHomePage()
    {
        WebDriver.Navigate().GoToUrl("https://example.com");
        WebDriver.CaptureScreenshot(TestObject, "home");        // attached to the result automatically
        await Assert.That(WebDriver.Title).IsNotEmpty();
    }
}

Without a test framework (runs from any host — console, worker, script):

using YAAQS.Selenium;
using YAAQS.Core.Logging;

ILogger log = LoggerFactory.GetLogger("automation");
using var a = new SeleniumObject(WebDriverFactory.GetDefaultBrowser, log, "Demo");
a.WebDriver.Navigate().GoToUrl("https://example.com");
a.WebDriver.CaptureScreenshot(a, "home");                       // saved to disk + embedded in the HTML log
log.Dispose();

Playwright + TUnit

With a test framework (Playwright is async):

using YAAQS.Playwright.Testing;

public class LoginTests : PlaywrightTest
{
    [Test]
    public async Task CanReachHomePage()
    {
        await Page.GotoAsync("https://example.com");
        await Page.CaptureScreenshotAsync(TestObject, "home");
        await Assert.That(await Page.TitleAsync()).IsNotEmpty();
    }
}

Without a test framework:

using YAAQS.Playwright;
using YAAQS.Core.Logging;

PlaywrightBrowserInstaller.EnsureInstalled("chromium");         // one-time, no-op if already present
ILogger log = LoggerFactory.GetLogger("automation");
PlaywrightGraph g = await PlaywrightDriverFactory.CreateDefaultAsync();
await using var a = new PlaywrightObject(g.Playwright, g.Browser, g.Context, g.Page, log, "Demo");
await a.Page.GotoAsync("https://example.com");
await a.Page.CaptureScreenshotAsync(a, "home");
log.Dispose();

Run: dotnet run --project tests/YAAQS.Selenium.TUnit.Tests -c Release --report-trx --report-trx-filename TestResults.trx

NUnit

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <EnableNUnitRunner>true</EnableNUnitRunner>
    <TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
    
    <RootNamespace>MyTests</RootNamespace>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="NUnit" Version="4.6.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="6.0.0" />
    <PackageReference Include="Microsoft.Testing.Extensions.TrxReport" Version="2.2.3" />
  </ItemGroup>
  <ItemGroup>
    
    <ProjectReference Include="..\..\src\YAAQS.Selenium.NUnit\YAAQS.Selenium.NUnit.csproj" />
  </ItemGroup>
</Project>

Selenium + NUnit

With a test framework:

using NUnit.Framework;
using YAAQS.Selenium.Testing;

public class LoginTests : SeleniumTest
{
    [Test]
    public void CanReachHomePage()
    {
        WebDriver.Navigate().GoToUrl("https://example.com");
        WebDriver.CaptureScreenshot(TestObject, "home");
        Assert.That(WebDriver.Title, Is.Not.Empty);
    }
}

Without a test framework: same as the standalone Selenium snippet above — unchanged by NUnit.

Playwright + NUnit

With a test framework:

using NUnit.Framework;
using YAAQS.Playwright.Testing;

public class LoginTests : PlaywrightTest
{
    [Test]
    public async Task CanReachHomePage()
    {
        await Page.GotoAsync("https://example.com");
        await Page.CaptureScreenshotAsync(TestObject, "home");
        Assert.That(await Page.TitleAsync(), Is.Not.Empty);
    }
}

Without a test framework: same as the standalone Playwright snippet above — unchanged by NUnit.

Run: dotnet run --project tests/YAAQS.Selenium.NUnit.Tests -c Release --report-trx --report-trx-filename TestResults.trx

MSTest

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <EnableMSTestRunner>true</EnableMSTestRunner>
    <TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="MSTest.TestAdapter" Version="4.2.3" />
    <PackageReference Include="MSTest.TestFramework" Version="4.2.3" />
    <PackageReference Include="Microsoft.Testing.Extensions.TrxReport" Version="2.2.3" />
  </ItemGroup>
  <ItemGroup>
    
    <ProjectReference Include="..\..\src\YAAQS.Selenium.MSTest\YAAQS.Selenium.MSTest.csproj" />
  </ItemGroup>
</Project>

MSTest requires [TestClass] on the fixture; the base supplies the TestContext the result sink needs.

Selenium + MSTest

With a test framework:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using YAAQS.Selenium.Testing;

[TestClass]
public class LoginTests : SeleniumTest
{
    [TestMethod]
    public void CanReachHomePage()
    {
        WebDriver.Navigate().GoToUrl("https://example.com");
        WebDriver.CaptureScreenshot(TestObject, "home");
        Assert.IsFalse(string.IsNullOrEmpty(WebDriver.Title));
    }
}

Without a test framework: same as the standalone Selenium snippet above — unchanged by MSTest.

Playwright + MSTest

With a test framework:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using YAAQS.Playwright.Testing;

[TestClass]
public class LoginTests : PlaywrightTest
{
    [TestMethod]
    public async Task CanReachHomePage()
    {
        await Page.GotoAsync("https://example.com");
        await Page.CaptureScreenshotAsync(TestObject, "home");
        Assert.IsFalse(string.IsNullOrEmpty(await Page.TitleAsync()));
    }
}

Without a test framework: same as the standalone Playwright snippet above — unchanged by MSTest.

Run: dotnet run --project tests/YAAQS.Selenium.MSTest.Tests -c Release --report-trx --report-trx-filename TestResults.trx

xUnit (v3)

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    
    <RootNamespace>MyTests</RootNamespace>
  </PropertyGroup>
  <ItemGroup>
    
    <PackageReference Include="xunit.v3" Version="3.2.2" />
  </ItemGroup>
  <ItemGroup>
    
    <ProjectReference Include="..\..\src\YAAQS.Selenium.Xunit\YAAQS.Selenium.Xunit.csproj" />
  </ItemGroup>
</Project>

Selenium + xUnit

With a test framework:

using Xunit;
using YAAQS.Selenium.Testing;

public class LoginTests : SeleniumTest
{
    [Fact]
    public void CanReachHomePage()
    {
        WebDriver.Navigate().GoToUrl("https://example.com");
        WebDriver.CaptureScreenshot(TestObject, "home");
        Assert.False(string.IsNullOrEmpty(WebDriver.Title));
    }
}

Without a test framework: same as the standalone Selenium snippet above — unchanged by xUnit.

Playwright + xUnit

With a test framework:

using Xunit;
using YAAQS.Playwright.Testing;

public class LoginTests : PlaywrightTest
{
    [Fact]
    public async Task CanReachHomePage()
    {
        await Page.GotoAsync("https://example.com");
        await Page.CaptureScreenshotAsync(TestObject, "home");
        Assert.False(string.IsNullOrEmpty(await Page.TitleAsync()));
    }
}

Without a test framework: same as the standalone Playwright snippet above — unchanged by xUnit.

Run: dotnet run --project tests/YAAQS.Selenium.Xunit.Tests -c Release -- -trx TestResults.trx

<a id="playwright-browsers"></a>

Install the Playwright browsers once. Unlike Selenium Manager, Playwright does not auto-resolve browsers at runtime. Run a one-time install before your Playwright tests/CI — programmatically via YAAQS.Playwright.PlaywrightBrowserInstaller.EnsureInstalled("chromium"), or the bundled pwsh bin/.../playwright.ps1 install (with OS deps on Linux: ... install --with-deps chromium). Keep PLAYWRIGHT_BROWSERS_PATH identical at install time and run time, do not install per-test, and throttle concurrency for large Playwright suites (browsers are heavy and frameworks parallelize by default).

For complete standalone programs — logging, the StandaloneResultSink that collects attachments when there's no runner, and error handling — see samples/YAAQS.Selenium.ConsoleSample and samples/YAAQS.Playwright.ConsoleSample.

Configuration

Settings come from appsettings.json next to the host app, environment variables, then programmatic overrides (Config.AddTestSettingValue). Global is shared; each technology has its own section:

{
  "Global": {
    "Log": "OnFail",           // Yes | No | OnFail (keep log only on failure)
    "LogLevel": "Information",  // Verbose | Information | Action | Step | Generic | Success | Warning | Error | Suspended
    "LogType": "HTML",          // TXT | HTML | CONSOLE
    "FileLoggerPath": "",       // default: <app>/Logs
    "Timeout": "10000",         // GenericWait timeout (ms)
    "WaitTime": "100"           // GenericWait poll interval (ms)
  },
  "Selenium": {
    "Browser": "Chrome",        // Chrome | HeadlessChrome | Edge | HeadlessEdge | Firefox | HeadlessFirefox | Safari | Remote
    "BrowserTimeout": "30000",  // required
    "BrowserSize": "MAXIMIZE",  // MAXIMIZE | DEFAULT | 1920x1080
    "SavePagesourceOnFail": "Yes",
    "SoftAssertScreenshot": "Yes"
  },
  "Playwright": {
    "Browser": "Chromium",      // Chromium | Chrome | Edge | Firefox | Webkit
    "Headless": "Yes",          // Yes | No
    "Timeout": "30000",         // page default timeout (ms)
    "BrowserSize": "1280x720",  // #x#, or DEFAULT
    "SavePageSourceOnFail": "Yes",
    "CaptureTrace": "No",       // record a Playwright trace.zip per test
    "CaptureVideo": "No",       // record a .webm per test
    "UseProxy": "No",
    "ProxyAddress": ""
  }
}

Every Playwright value has a default, so it works with no configuration at all. Technology-specific section names live with their library (SeleniumConfigSection / PlaywrightConfigSection), so the core never carries another technology's keys.

Azure DevOps test attachments

Every adapter runs on Microsoft.Testing.Platform and produces a TRX whose <ResultFile> entries carry the screenshots/logs (and Playwright trace/video). How you ask for the TRX depends on the framework:

# TUnit / NUnit / MSTest (MTP TrxReport extension — flows in transitively for TUnit; add the package for NUnit/MSTest)
dotnet run -c Release --report-trx --report-trx-filename TestResults.trx --results-directory $(Agent.TempDirectory)

# xUnit v3 (in-process runner writes TRX natively, attachments included — no extension needed)
dotnet run -c Release -- -trx TestResults.trx

Then publish (identical for all four — it just consumes a TRX):

- task: PublishTestResults@2
  condition: always()
  inputs:
    testResultsFormat: 'VSTest'          # 'VSTest' == TRX (correct for MTP-produced TRX)
    testResultsFiles: '**/TestResults*.trx'
    searchFolder: '$(Agent.TempDirectory)'
    mergeTestResults: true
    publishRunAttachments: true          # uploads the attached screenshots/logs

For portability across agents/containers, point FileLoggerPath at a folder inside --results-directory so attachment paths travel with the TRX.

Migration notes from MAQS

Intentional scope changes: Internet Explorer support was dropped (EOL); accessibility (Selenium.Axe) was omitted (maintained Deque.AxeCore.Selenium is not yet compatible with Selenium 4.45). Playwright is async-native — the MAQS synchronous PageDriver/PlaywrightSyncElement wrappers are intentionally not reproduced; capture is screenshot + page source on failure, with optional trace/video via config.

Building and testing

dotnet build YAAQS.slnx -c Release

# Selenium suites (Core + lifecycle + Selenium)
dotnet run --project tests/YAAQS.Selenium.TUnit.Tests  -c Release   # + NUnit / MSTest / Xunit

# Playwright suites (install browsers once first)
#   PlaywrightBrowserInstaller.EnsureInstalled("chromium")  — or pwsh playwright.ps1 install chromium
dotnet run --project tests/YAAQS.Playwright.TUnit.Tests -c Release   # + NUnit / MSTest / Xunit

Project layout

src/YAAQS.Core               runner- & technology-agnostic foundation (logging, config, waits, object model,
                             soft assert, AutomationLifecycle sync+async, IResultSink — no technology, no test fwk)
src/YAAQS.Selenium           Selenium automation library (driver factory, SeleniumObject, capture, page model,
                             SeleniumLifecycle — no test framework)
src/YAAQS.Playwright         Playwright automation library (PlaywrightDriverFactory, PlaywrightObject, async
                             PlaywrightLifecycle, capture pipeline, browser installer — no test framework)
src/YAAQS.<Fw>               shared, technology-free framework binding (AutomationTest + <Fw>ResultSink),
                             one per TUnit / NUnit / MSTest / Xunit
src/YAAQS.Selenium.<Fw>      Selenium leaf adapter  (SeleniumTest)   — refs YAAQS.<Fw> + YAAQS.Selenium
src/YAAQS.Playwright.<Fw>    Playwright leaf adapter (PlaywrightTest) — refs YAAQS.<Fw> + YAAQS.Playwright
tests/YAAQS.<Tech>.<Fw>.Tests          one validation suite per technology × framework
samples/YAAQS.<Tech>.ConsoleSample     general automation from a console app (no test framework)