TUnit.Engine 0.1.504

TUnit

T(est)Unit!

Documentation

See here: https://thomhurst.github.io/TUnit/

Speed Comparison

We can't get an extremely accurate speed comparison because the test host runs as its own process - So tools like Benchmark.NET won't work.

But I've tried to create 4 equal basic test suites, which all execute a test method with a 50ms Task.Delay, and repeat it 1000 times. Some frameworks don't easily have repeat mechanisms so this may be artificially produced with a test data source.

While a 50ms Task.Delay isn't obviously a test in itself, it still helps to demonstrate how TUnit is fast at runtime, due to using a newer test architecture, source generation, parallelization and async-await all the way down the stack.

See here for the latest speed test comparisons: https://github.com/thomhurst/TUnit/actions/workflows/speed-comparison.yml?query=branch%3Amain

IDE

TUnit is built on top of newer Microsoft.Testing.Platform libraries, as opposed to older legacy VSTest libraries. As of July 2024, IDEs do not fully support this testing platform yet.

Visual Studio Preview versions can run the new tests by enabling the new testing platform server mode, within Visual Studio preview/experimental features. You will have to opt in to this manually.

For Rider, it is not yet supported. I believe they are working on it so we just have to wait for now.

dotnet CLI - Fully supported. Tests should be runnable with both dotnet test or dotnet run. dotnet run should give you a better experience and make it simpler to pass in test flags!

Features

• Source generated tests
• Full async support
• Parallel by default, with mechanisms to:
  • Run specific tests completely on their own
  • Run specific tests not in parallel with other specific tests
  • Limit the a parallel limit on a per-test, class or assembly level
• Test ordering (if running not in parallel)
• Tests can depend on other tests to form chains, useful for if one test depends on state from another action
• Easy to read assertions
• Injectable test data via classes, methods, compile-time args, or matrices
• Hooks before and after:
  • TestDiscover
  • TestSession
  • Assembly
  • Class
  • Test
• Designed to avoid common pitfalls such as leaky test states
• Ability to view and interrogate metadata and results from various assembly/class/test context objects

Installation

dotnet add package TUnit --prerelease

Example test

    [Test]
    public async Task Test1()
    {
        var value = "Hello world!";

        await Assert.That(value)
            .Is.Not.Null
            .And.Does.StartWith("H")
            .And.Has.Count().EqualTo(12)
            .And.Is.EqualTo("hello world!", StringComparison.InvariantCultureIgnoreCase);
    }

or with more complex test orchestration needs

    [Before(Class)]
    public static async Task ClearDatabase(ClassHookContext context) { ... }

    [After(Class)]
    public static async Task AssertDatabaseIsAsExpected(ClassHookContext context) { ... }

    [Before(Test)]
    public async Task CreatePlaywrightBrowser(TestContext context) { ... }

    [After(Test)]
    public async Task DisposePlaywrightBrowser(TestContext context) { ... }

    [Retry(3)]
    [Test, DisplayName("Register an account")]
    [EnumerableMethodData(nameof(GetAuthDetails))]
    public async Task Register(string username, string password) { ... }

    [Repeat(5)]
    [Test, DependsOn(nameof(Register))]
    [EnumerableMethodData(nameof(GetAuthDetails))]
    public async Task Login(string username, string password) { ... }

    [Test, DependsOn(nameof(Login), [typeof(string), typeof(string)])]
    [EnumerableMethodData(nameof(GetAuthDetails))]
    public async Task DeleteAccount(string username, string password) { ... }

    [Category("Downloads")]
    [Timeout(300_000)]
    [Test, NotInParallel(Order = 1)]
    public async Task DownloadFile1() { ... }

    [Category("Downloads")]
    [Timeout(300_000)]
    [Test, NotInParallel(Order = 2)]
    public async Task DownloadFile2() { ... }

    [Repeat(10)]
    [Test]
    [Arguments(1)]
    [Arguments(2)]
    [Arguments(3)]
    [DisplayName("Go to the page numbered $page")]
    public async Task GoToPage(int page) { ... }

    [Category("Cookies")]
    [Test, Skip("Not yet built!")]
    public async Task CheckCookies() { ... }

    [Test, Explicit, WindowsOnlyTest, RetryHttpServiceUnavailable(5)]
    [Property("Some Key", "Some Value")]
    public async Task Ping() { ... }

    [Test]
    [ParallelLimit<LoadTestParallelLimit>]
    [Repeat(1000)]
    public async Task LoadHomepage() { ... }

    public static IEnumerable<(string Username, string Password)> GetAuthDetails()
    {
        yield return ("user1", "password1");
        yield return ("user2", "password2");
        yield return ("user3", "password3");
    }

    public class WindowsOnlyTestAttribute : SkipAttribute
    {
        public WindowsOnlyTestAttribute() : base("Windows only test")
        {
        }

        public override Task<bool> ShouldSkip(TestContext testContext)
        {
            return Task.FromResult(!OperatingSystem.IsWindows());
        }
    }

    public class RetryHttpServiceUnavailableAttribute : RetryAttribute
    {
        public RetryHttpServiceUnavailableAttribute(int times) : base(times)
        {
        }

        public override Task<bool> ShouldRetry(TestInformation testInformation, Exception exception, int currentRetryCount)
        {
            return Task.FromResult(exception is HttpRequestException { StatusCode: HttpStatusCode.ServiceUnavailable });
        }
    }

    public class LoadTestParallelLimit : IParallelLimit
    {
        public int Limit => 50;
    }

Motivations

TUnit is inspired by NUnit and xUnit - two of the most popular testing frameworks for .NET.

It aims to build upon the useful features of both while trying to address any pain points that they may have. You may have experienced these, or you may have not even known about them.

Read more here