ReqnrollToElasticReporter.NUnit 1.0.0

Reqnroll to Elasticsearch Reporter

A robust, production-ready library for reporting Reqnroll (formerly SpecFlow) test automation results to Elasticsearch. This library provides real-time test result tracking with background processing, retry logic, and comprehensive test metadata.

1. Setup of the Test Environment

To use this reporter, you need a running instance of Elasticsearch and Kibana. The easiest way to set this up is using Docker Compose.

Docker Compose Setup

Create a docker-compose.yml file in your solution root:

version: '3.8'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.1
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ports:
      - "9200:9200"
    networks:
      - elastic-net

  kibana:
    image: docker.elastic.co/kibana/kibana:8.11.1
    container_name: kibana
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch
    networks:
      - elastic-net

networks:
  elastic-net:
    driver: bridge

Start the services:

docker-compose up -d

• Elasticsearch: http://localhost:9200
• Kibana: http://localhost:5601

2. Installation

Add the NuGet package to your test project:

dotnet add package ReqnrollToElasticReporter.NUnit

Or via Package Manager Console:

Install-Package ReqnrollToElasticReporter.NUnit

3. Usage

Configuration (appsettings.json)

Add an Elasticsearch section to your appsettings.json. Ensure the file is set to Copy to Output Directory.

{
  "Elasticsearch": {
    "Enabled": true,
    "Url": "http://localhost:9200",
    "Index": "automation-tests",
    "Username": "", 
    "Password": "",
    "ApiKey": "",
    "BatchSize": 10,
    "FlushInterval": 5000,
    "MaxRetries": 3,
    "RetryDelayMs": 1000
  }
}

Hooks Setup

Create a Hooks/ElasticsearchConfigurationHooks.cs file to initialize the reporter. This is where you configure how screenshots are taken and how resources are cleaned up.

using System;
using System.IO;
using System.Threading.Tasks;
using Reqnroll;
using ReqnrollToElasticReporter.Reporting.Elasticsearch;
using ReqnrollToElasticReporter.Reporting.Elasticsearch.Configuration;
using ReqnrollToElasticReporter.Reporting.Elasticsearch.Models;
using Microsoft.Playwright; // Required if using Playwright

namespace MyTestSuite.Hooks;

[Binding]
public class ElasticsearchConfigurationHooks
{
    [BeforeTestRun(Order = 0)]
    public static void InitializeElasticsearchReporting()
    {
        Console.WriteLine("=== Initializing Elasticsearch Reporting ===");

        // 1. Load Configuration
        var config = ElasticsearchConfig.Load();
        
        // Exit if disabled
        if (!config.Enabled) 
        {
            Console.WriteLine("Elasticsearch reporting is disabled.");
            return;
        }

        // 2. Create Run Metadata
        // This info attaches to every test result in this run
        var runInfo = new TestRunInfo
        {
            RunId = Guid.NewGuid().ToString(),
            StartTime = DateTime.UtcNow,
            Environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? "Development",
            MachineName = Environment.MachineName,
            UserName = Environment.UserName,
            PipelineId = Environment.GetEnvironmentVariable("CI_PIPELINE_ID")
        };

        // 3. Initialize the Background Service
        BackgroundReportingService.Instance.Initialize(config, runInfo);

        // 4. Configure Screenshot Provider
        // The reporter calls this when a test fails.
        ElasticsearchReporting.ScreenshotProvider = async (scenarioContext) =>
        {
            try 
            {
                // Check if Playwright Page object is stored in ScenarioContext
                // Ensure your tests set _scenarioContext["Page"] = page;
                if (scenarioContext.TryGetValue("Page", out var pageObj) && pageObj is IPage page)
                {
                    var screenshotDir = Path.Combine(Directory.GetCurrentDirectory(), "test-results", "screenshots");
                    Directory.CreateDirectory(screenshotDir);
                    
                    // Create a safe filename based on scenario title and timestamp
                    var safeTitle = scenarioContext.ScenarioInfo.Title.Replace(" ", "_");
                    var fileName = $"{safeTitle}_{DateTime.Now:yyyyMMddHHmmss}.png";
                    var screenshotPath = Path.Combine(screenshotDir, fileName);
                    
                    await page.ScreenshotAsync(new PageScreenshotOptions 
                    { 
                        Path = screenshotPath, 
                        FullPage = true 
                    });
                    
                    Console.WriteLine($"[Elasticsearch] Screenshot saved: {screenshotPath}");
                    return screenshotPath;
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine($"[Elasticsearch] Failed to capture screenshot: {ex.Message}");
            }
            return null;
        };
        
        // 5. Configure Resource Cleaner
        // The reporter calls this after processing the scenario
        ElasticsearchReporting.ResourceCleaner = async (scenarioContext) =>
        {
             try
             {
                 if (scenarioContext.TryGetValue("Page", out var pageObj) && pageObj is IPage page)
                 {
                     await page.CloseAsync();
                 }
                 
                 if (scenarioContext.TryGetValue("Context", out var contextObj) && contextObj is IBrowserContext context)
                 {
                     await context.CloseAsync();
                 }
             }
             catch (Exception ex)
             {
                 Console.WriteLine($"[Elasticsearch] Error cleaning up resources: {ex.Message}");
             }
        };
    }
}

4. Test Results Schema

The reporter sends JSON documents to Elasticsearch with the following structure:

• Id: Unique deterministic ID (PipelineId_ScenarioName)
• RunId: Unique ID for the test run
• Status: Passed, Failed, Skipped
• TestName / ScenarioName: Name of the test
• FeatureName: Name of the feature file
• Steps: Array of step details
  • StepText: "Given I am logged in"
  • Status: Passed/Failed
  • Duration: Execution time in seconds
  • ErrorMessage: If failed
• Duration: Total execution time
• Environment: QA, Staging, etc.
• Browser: chrome, firefox (if tagged)
• ScreenshotPath: Path to failure screenshot
• Error: Exception message and stack trace

5. Architecture

• BackgroundReportingService: A singleton service that manages a thread-safe queue of test results. It processes results in batches to minimize network calls and ensure test execution is not blocked.
• ElasticsearchReporter: Handles the low-level communication with Elasticsearch using the Elastic.Clients.Elasticsearch library. It implements retry logic using Polly.
• ElasticsearchReportingHooks: Automatically registered Reqnroll hooks that capture test lifecycle events (Start, Step, Finish) and enqueue results.

6. Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository.
2. Create a feature branch.
3. Commit your changes.
4. Open a Pull Request.

Please ensure all unit tests pass before submitting.

7. License

This project is licensed under the MIT License - see the LICENSE file for details.

8. Support

For issues, questions, or feature requests, please open an issue on GitHub.