Bromine.Xray.Reporter 1.0.0

Bromine.Xray.Reporter

Framework-agnostic .NET library for reporting BDD/test results to Xray Cloud. Collects results during your test run and batch-uploads them as a Test Execution at the end. Works with Reqnroll, SpecFlow, NUnit, xUnit, MSTest, or anything else that can produce (testKey, status) pairs.

Features

• Zero coupling to a specific test framework
• Find or create long-lived Test Plans and Test Executions by label
• Per-test timing, retries, and ScenarioOutline example tracking
• Attach screenshots/files as evidence from anywhere in your test
• ILogger integration (or silent by default)
• CancellationToken support throughout
• Automatic retry with exponential backoff on 429 and 5xx (honors Retry-After)
• Thread-safe — safe for parallel test execution

Install

dotnet add package Bromine.Xray.Reporter

Quick start

using Bromine.Xray.Reporter;
using Bromine.Xray.Reporter.Models;

var reporter = new XrayTestReporter(new XrayOptions
{
    ClientId = Environment.GetEnvironmentVariable("XRAY_CLIENT_ID")!,
    ClientSecret = Environment.GetEnvironmentVariable("XRAY_CLIENT_SECRET")!,
    ProjectKey = "PROJ",
    Jira = new JiraOptions
    {
        BaseUrl = "https://yoursite.atlassian.net",
        Email = Environment.GetEnvironmentVariable("JIRA_EMAIL"),
        ApiToken = Environment.GetEnvironmentVariable("JIRA_API_TOKEN")
    }
});

// Find-or-create Test Plan + Test Execution, both keyed by label
await reporter.StartExecutionAsync(
    summary: "Nightly_Smoke_Tests",
    environment: "Staging",
    plan: new TestPlanRef(
        summary: "Smoke Test Coverage",
        label: "smoke-coverage"),
    execution: new TestExecutionRef(
        label: "nightly-smoke-staging"));

// Record results during your test run
reporter.StartTracking("PROJ-101");
// ... run the test ...
reporter.RecordResult(new XrayTestResult
{
    TestKey = "PROJ-101",
    Status = XrayTestStatus.Passed
});

// Submit at the end
var response = await reporter.SubmitResultsAsync();
Console.WriteLine($"Test Execution: {response?.Key}");

Reqnroll / SpecFlow integration

using Bromine.Xray.Reporter;
using Bromine.Xray.Reporter.Extensions;
using Bromine.Xray.Reporter.Models;

[Binding]
public class XrayHooks(ScenarioContext scenarioContext)
{
    private static XrayTestReporter _reporter = null!;

    [BeforeTestRun]
    public static async Task BeforeTestRun()
    {
        _reporter = new XrayTestReporter(new XrayOptions
        {
            ClientId = Environment.GetEnvironmentVariable("XRAY_CLIENT_ID")!,
            ClientSecret = Environment.GetEnvironmentVariable("XRAY_CLIENT_SECRET")!,
            ProjectKey = "PROJ",
            Jira = new JiraOptions
            {
                BaseUrl = "https://yoursite.atlassian.net",
                Email = Environment.GetEnvironmentVariable("JIRA_EMAIL"),
                ApiToken = Environment.GetEnvironmentVariable("JIRA_API_TOKEN")
            }
        });

        await _reporter.StartExecutionAsync(
            summary: $"Nightly Run {DateTime.UtcNow:yyyy-MM-dd}",
            environment: "Staging",
            plan: new TestPlanRef("Smoke Test Coverage", "smoke-coverage"),
            execution: new TestExecutionRef("nightly-smoke-staging"));
    }

    [BeforeScenario]
    public void BeforeScenario()
    {
        var key = scenarioContext.ScenarioInfo.Tags.GetXrayKey();
        if (key is not null)
            _reporter.StartTracking(key);
    }

    [AfterScenario]
    public void AfterScenario()
    {
        var key = scenarioContext.ScenarioInfo.Tags.GetXrayKey();
        if (key is null) return;

        var error = scenarioContext.TestError;
        _reporter.RecordResult(new XrayTestResult
        {
            TestKey = key,
            Status = error is not null ? XrayTestStatus.Failed : XrayTestStatus.Passed,
            Comment = error?.Message
        }, label: GetScenarioOutlineLabel());
    }

    [AfterTestRun]
    public static async Task AfterTestRun()
    {
        var result = await _reporter.SubmitResultsAsync();
        _reporter.Dispose();
    }

    private string? GetScenarioOutlineLabel()
    {
        var args = scenarioContext.ScenarioInfo.Arguments;
        if (args is null || args.Count == 0) return null;
        return string.Join(", ", args.Keys.Cast<string>().Select(k => $"{k}: {args[k]}"));
    }
}

Tag your scenarios with @Key:PROJ-42 (or use a custom prefix — see Tag format).

Tag format

Default: @Key:PROJ-42. The prefix is configurable:

tags.GetXrayKey()            // matches @Key:PROJ-42
tags.GetXrayKey("TestId")    // matches @TestId:PROJ-42
tags.GetXrayKeys("Xray")     // matches @Xray:PROJ-312, @Xray:PROJ-313 (multi-key)

Appending text to a test's comment

Use AppendComment(testKey, text) to incrementally build up the comment for a test from anywhere during the run — step-by-step progress, log lines, anything that should end up in the Xray Test Run's comment field. Buffered per test key, drained into the result when RecordResult is called.

private DateTime _stepStart;

[BeforeStep]
public void TrackStep() => _stepStart = DateTime.UtcNow;

[AfterStep]
public void RecordStepResult()
{
    var key = scenarioContext.ScenarioInfo.Tags.GetXrayKey();
    if (key is null) return;

    var info = scenarioContext.StepContext.StepInfo;
    var duration = DateTime.UtcNow - _stepStart;
    var err = scenarioContext.TestError;
    var status = err is null ? XrayTestStatus.Passed : XrayTestStatus.Failed;

    var line = $"- {info.StepDefinitionType} {info.Text}: {status} ({duration:mm\\:ss})";
    if (err is not null) line += $" — {err.Message}";

    _reporter.AppendComment(key, line);
}

The comment for the test will look like:

FAILED (00:45)
- Given I am logged in: PASSED (00:02)
- When I click the button: PASSED (00:01)
- Then I see the dialog: FAILED (00:42) — Element not found: #dialog

This works with any framework — the library is just a string buffer with a flush-on-RecordResult lifecycle. Reqnroll, SpecFlow, raw NUnit, anything with before/after hooks works.

Attaching evidence

Attach screenshots or files from anywhere during a scenario — they're merged into the result on RecordResult:

var bytes = screenshot.AsByteArray;
_reporter.AttachEvidence(key, Convert.ToBase64String(bytes), "screenshot.png");
_reporter.AttachEvidence(key, Convert.ToBase64String(logBytes), "test.log", XrayContentType.PlainText);

ScenarioOutline / parameterized tests

Pass a label to RecordResult to distinguish example rows:

_reporter.RecordResult(new XrayTestResult
{
    TestKey = "PROJ-42",
    Status = XrayTestStatus.Passed
}, label: "user_role: admin");

When the same testKey is recorded multiple times with labels:

• All attempts are preserved with timing and evidence
• Overall status is PASSED only if every example row's last attempt passed (retries on failed rows are respected)
• The comment includes a structured breakdown of all rows

Test Plan / Test Execution reuse

StartExecutionAsync with TestPlanRef and TestExecutionRef finds existing issues by label and reuses them, or creates new ones if not found. This lets you maintain long-lived Test Plans/Executions that accumulate history across runs.

await reporter.StartExecutionAsync(
    summary: "Nightly Smoke - 2026-06-04",
    plan: new TestPlanRef("Smoke Coverage", "smoke-coverage"),       // find or create by label
    execution: new TestExecutionRef("nightly-smoke-staging"));        // find or create by label

On reuse:

• The execution's summary and description are updated each run
• Test Plan and Test Environments are set once at creation (Xray Cloud limitation — these custom fields aren't on Jira's edit screen and can't be updated after the fact)

Logging

Pass an ILogger<XrayTestReporter> to see what's happening:

var loggerFactory = LoggerFactory.Create(b => b.AddConsole());
var reporter = new XrayTestReporter(options, loggerFactory.CreateLogger<XrayTestReporter>());

Without a logger, the library is silent.

Cancellation

All async methods accept a CancellationToken:

using var cts = new CancellationTokenSource(TimeSpan.FromMinutes(5));
await reporter.SubmitResultsAsync(cts.Token);

Cancellation propagates cleanly through retries and HTTP calls.

API reference

XrayTestReporter

XrayTestStatus (string constants)

These constants are the Xray Cloud defaults. Your Xray admin can rename them (e.g. "PASSED" → "PASS") or add custom ones (e.g. "PARTIALLY_PASSED") per project. The library accepts any string — Xray itself validates the value on import.

// Use a built-in constant
reporter.RecordResult(new XrayTestResult { TestKey = "PROJ-1", Status = XrayTestStatus.Passed });

// Or any custom string your project supports
reporter.RecordResult(new XrayTestResult { TestKey = "PROJ-1", Status = "PARTIALLY_PASSED" });

If your project renames the success/failure statuses, configure them on XrayOptions so the ScenarioOutline overall-status resolver works correctly:

var options = new XrayOptions
{
    // ... credentials ...
    PassedStatus = "PASS",   // default: "PASSED"
    FailedStatus = "FAIL"    // default: "FAILED"
};

XrayContentType (enum)

Strict enum for evidence MIME types — consumers can't pass arbitrary strings.

XrayOptions

JiraOptions

Permissions

The Xray client ID/secret needs:

• Read/write access to test executions in the configured project

The Jira API token needs:

• Read access to issues (JQL search by label)
• Create issues (for Test Plans / first-time Test Executions)
• Edit issues (summary, description, labels, parent)
• Create issue links (Test Plan ↔ Test Execution)

Xray Cloud constraints

• Status values must match your project's Xray status configuration
• Test Plan and Test Environments custom fields can't be updated on existing executions through the import endpoint — they're set at creation only
• The v2 import/execution endpoint does not support iterations or labels in the info block — this library handles both via Jira REST and structured comments instead

Documentation

• Xray Cloud REST API v2 — Import Execution Results
• Xray Cloud Authentication
• Jira Cloud REST API v3

Contributing

PRs welcome. To build and test locally:

dotnet build
dotnet test

License

MIT — see LICENSE.