Gabonet.Hubble 0.4.2

Hubble for .NET

Hubble is an embedded monitoring and logging dashboard for ASP.NET Core applications, in the spirit of Laravel Telescope. It captures HTTP requests and responses, ILogger messages and database queries, correlates them per request, stores them in MongoDB and shows them in a built-in web UI — no extra service to deploy.

• Features
• Requirements
• Quick start
• Configuration
• Securing the dashboard
• Capturing ILogger messages
• Background jobs (Hangfire)
• Capturing database queries
• Tagging endpoints
• Searching
• Files in requests and responses
• Selective capture (production)
• Masking sensitive data
• How capture works (and what happens if MongoDB is down)
• Data retention and MongoDB setup
• Healthy implementation checklist
• Web UI and REST API
• Building from source
• Troubleshooting
• Known limitations
• Upgrade notes

Features

• HTTP capture: method, path, query string, headers, request and response bodies, status code and duration.
• ILogger correlation: log messages written while a request is being processed are linked to that request and shown in its detail page.
• Background jobs: with the optional Gabonet.Hubble.Hangfire package, a Jobs section lists Hangfire's recurring jobs (cron, time zone, last and next execution, status), the history of every job and the ILogger messages written during each execution.
• Database queries: SQL captured through Entity Framework Core interceptors or ADO.NET, plus an example for MongoDB command monitoring.
• Tags: label endpoints with [HubbleTag], WithHubbleTags, HttpContext.AddHubbleTag or by path, and filter the dashboard by them.
• Multi-field search: one search box over URL, request and response bodies, error messages and tags (auth/login + p2payservice); your application chooses the searchable fields.
• Files: uploads and binary responses are stored as files that can be viewed (PDF and images open in a new tab) or downloaded, instead of being rendered as text.
• Selective capture: in production, store only failures and/or tagged requests.
• Data masking: case-insensitive, recursive masking of JSON properties and headers, with a helper for your own logs.
• Built-in dashboard: a side menu to jump between requests, ILogger messages, errors and database queries; filtering, search, a quick preview of request and response bodies from the list, pagination, a statistics page and a JSON REST API.
• Dashboard security: optional login with a signed session cookie, CSRF protection, brute-force lockout, IP/CIDR allow-list and hardened response headers.
• Non-blocking capture: logs are queued in memory and written to MongoDB in the background, so a slow or unavailable MongoDB never slows down or breaks your application.
• Retention: optional automatic pruning of old logs.

Requirements

Component	Version
Host application	ASP.NET Core on .NET 6, 7, 8 or later (the package targets net6.0, net7.0 and net8.0; .NET 9+ uses the net8.0 build)
Storage	A MongoDB server supported by MongoDB.Driver 3.2
Optional	Entity Framework Core for SQL capture (the package depends on EF Core 5.0.17+ on net6.0, 7 on net7.0 and 8 on net8.0)
Optional	Hangfire 1.7.11 or later (1.7.x and 1.8.x) for the Jobs section, through the separate Gabonet.Hubble.Hangfire package

Quick start

1. Install the package

dotnet add package Gabonet.Hubble

2. Register and enable Hubble

using Gabonet.Hubble.Extensions;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHubble(options =>
{
    options.ConnectionString = builder.Configuration.GetConnectionString("Hubble")!; // required
    options.DatabaseName = "HubbleDB";                                                 // required
    options.ServiceName = "Orders.Api";

    // Never monitor health checks, metrics, docs or streaming endpoints (see "Known limitations")
    options.IgnorePaths = new List<string> { "/health", "/metrics", "/swagger" };

    // Protect the dashboard (credentials come from configuration / secrets, never source code)
    options.RequireAuthentication = true;
    options.Username = builder.Configuration["Hubble:Username"]!;
    options.Password = builder.Configuration["Hubble:Password"]!;
});

// Optional: capture ILogger messages and link them to the current request
builder.Logging.AddHubbleLogging(LogLevel.Information);

var app = builder.Build();

app.UseHubble(); // before UseRouting() so that every request is captured

app.UseRouting();
app.MapControllers();

app.Run();

3. Open the dashboard

Browse to https://your-app/hubble (or the BasePath you configured).

Middleware order

The position of UseHubble() matters:

var app = builder.Build();

// 1. Behind a reverse proxy / load balancer: restore the real client IP first,
//    otherwise AllowedIps and the login lockout only ever see the proxy's address.
app.UseForwardedHeaders();

// 2. Global exception handler (optional). Placed before Hubble, it still receives the
//    exception after Hubble has recorded it, because Hubble re-throws.
app.UseExceptionHandler("/error");

// 3. Hubble: capture middleware + dashboard.
app.UseHubble();

// 4. The rest of your pipeline.
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
app.MapControllers();

To trust forwarded headers safely, tell ASP.NET Core which proxies are yours:

builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
    options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
    options.KnownProxies.Add(IPAddress.Parse("10.0.0.10")); // your proxy
});

Configuration

Hubble can be configured in code or from appsettings.json.

In code

builder.Services.AddHubble(options =>
{
    options.ConnectionString = "mongodb://localhost:27017";
    options.DatabaseName = "HubbleDB";
    options.ServiceName = "Orders.Api";
    options.BasePath = "/hubble";
    options.TimeZoneId = "America/New_York";
    options.IgnorePaths = new List<string> { "/health", "/metrics" };

    options.EnableDataPrune = true; // MongoDB deletes logs older than MaxLogAgeHours (TTL index)
    options.MaxLogAgeHours = 72;

    options.RequireAuthentication = true;
    options.Username = builder.Configuration["Hubble:Username"]!;
    options.Password = builder.Configuration["Hubble:Password"]!;
    options.AllowDeleteAll = false;

    options.Security.MaskBodyProperties = new List<string> { "password", "token", "creditCard", "cvv" };
    options.Security.MaskRequestBodyProperties = new List<string> { "pin" };
    options.Security.MaskResponseBodyProperties = new List<string> { "internalId" };
    options.Security.MaskHeaders = new List<string> { "Authorization", "Cookie", "X-Api-Key" };
    options.Security.AllowedIps = new List<string> { "127.0.0.1", "10.0.0.0/8" };
});

From appsettings.json

{
  "ConnectionStrings": {
    "Hubble": "mongodb://hubble_user@mongo:27017/HubbleDB"
  },
  "Hubble": {
    "ServiceName": "Orders.Api",
    "BasePath": "/hubble",
    "RequireAuthentication": true,
    "Username": "admin",
    "IgnorePaths": [ "/health", "/metrics", "/swagger" ],
    "EnableDataPrune": true,
    "MaxLogAgeHours": 72,
    "AllowDeleteAll": false,
    "Security": {
      "MaskBodyProperties": [ "password", "token", "creditCard", "cvv" ],
      "MaskRequestBodyProperties": [ "pin" ],
      "MaskResponseBodyProperties": [ "internalId" ],
      "MaskHeaders": [ "Authorization", "Cookie", "X-Api-Key" ],
      "AllowedIps": [ "10.0.0.0/8" ]
    }
  }
}

builder.Services.AddHubble(
    builder.Configuration.GetSection("Hubble"),
    options =>
    {
        // Runs after the section is read: complete or override values here
        options.ConnectionString = builder.Configuration.GetConnectionString("Hubble")!;
        options.DatabaseName = "HubbleDB";
    });

Lists in the section (IgnorePaths, Security.MaskHeaders, ...) replace the defaults instead of being appended to them. Every setting can also be placed in the section itself ("ConnectionString", "DatabaseName"), in which case the second argument is optional.

HubbleOptions is also registered as IOptions<HubbleOptions> if you need to read the effective configuration elsewhere.

Keep the password out of the file. ASP.NET Core does not expand ${VARIABLES} inside JSON; provide it through user secrets, a vault, or an environment variable using __ as the section separator:

export Hubble__Password="a-long-random-password"

Options reference

Option	Default	Description
ConnectionString	—	Required. MongoDB connection string. Unless it sets serverSelectionTimeoutMS, Hubble uses a 5-second server selection timeout.
DatabaseName	—	Required. Database that stores the HubbleLogs, HubbleStats and HubbleConfig collections (and HubbleJobRuns with the Jobs module).
ServiceName	HubbleService	Name stored with every log entry.
BasePath	/hubble	Dashboard route. Matched by path segment (/hubble does not match /hubblefoo). Cannot be empty or /.
PrefixPath	""	Prefix prepended to dashboard links and the session cookie path when the app is served under a sub-path by a proxy.
TimeZoneId	"" (UTC)	Time zone used to display timestamps (TimeZoneInfo id).
IgnorePaths	[]	Path prefixes that are never captured.
IgnoreStaticFiles	true	Skip common static file extensions.
CaptureHttpRequests	true	Capture HTTP traffic. When false, requests pass through untouched.
CaptureLoggerMessages	true	Capture ILogger messages (requires AddHubbleLogging()).
MinimumLogLevel	Information	Minimum ILogger level captured, unless AddHubbleLogging(level) sets one explicitly.
EnableDiagnostics	false	Write Hubble's internal errors to the console.
EnableDataPrune	false	Automatic retention: MongoDB deletes logs older than MaxLogAgeHours through a TTL index managed by Hubble.
MaxLogAgeHours	24	Retention in hours when EnableDataPrune is on (1 to 596,523). The app fails at startup if it is out of range.
DataPruneIntervalHours	1	No longer used (MongoDB checks for expired logs every minute). Kept for compatibility.
HighlightNewServices	false	Auto-refresh the list and highlight new entries.
HighlightDurationSeconds	5	How long new entries stay highlighted.
RequireAuthentication	false	Require a login for the dashboard and API. The app refuses to start if enabled with an empty Username or Password.
Username / Password	""	Dashboard credentials.
AllowDeleteAll	true	Allow deleting every log from the UI and API.
TagPaths	{}	Tags assigned by path: { "SAP": [ "/api/sap", "/api/*/sap/*" ] }. See Tagging endpoints.
SearchFields	Default	Fields the dashboard search looks in. See Searching.
CaptureOnly.MinimumStatusCode	null	Selective capture: store requests with this status code or higher. See Selective capture.
CaptureOnly.StatusCodes	[]	Selective capture: store requests with these exact status codes.
CaptureOnly.Tags	[]	Selective capture: store requests with any of these tags ("*" = any tagged request).
CaptureOnly.MinimumDurationMs	null	Selective capture: store requests that take this long or longer.
CaptureFileContents	true	Store the contents of uploaded and downloaded files so they can be viewed or downloaded from the dashboard.
MaxFileCaptureBytes	5242880	Maximum size of a file whose contents are stored (0 to 12 MB). Larger files keep only their metadata.
Jobs.Enabled	true	Jobs module on or off when AddHubbleHangfire() / AddHubbleJobs() is called. See Background jobs.
Jobs.CaptureArguments	true	Store the (masked) arguments of each job execution.
Security.MaskBodyProperties	see source	JSON properties masked in request and response bodies.
Security.MaskRequestBodyProperties	[]	Extra properties masked only in request bodies.
Security.MaskResponseBodyProperties	[]	Extra properties masked only in response bodies.
Security.MaskHeaders	Authorization, X-Api-Key, Cookie	Request headers stored as *****.
Security.AllowedIps	[]	IPs / CIDR ranges (IPv4 and IPv6) allowed to reach the dashboard and API. Empty or * allows everyone. Never affects the rest of your application.

Invalid settings make the application fail at startup with a single message listing every problem (missing ConnectionString or DatabaseName, authentication without credentials, an unknown TimeZoneId, an out-of-range MaxLogAgeHours, ...), instead of running with a broken or insecure configuration.

Securing the dashboard

Hubble stores request bodies, headers, SQL and log messages: treat the dashboard as an administrative tool that exposes sensitive data.

What Hubble enforces

• Authentication (RequireAuthentication = true): an HTML login form for browsers and HTTP Basic authentication for API clients. Credentials are compared in constant time.
• Signed session cookie: the HubbleAuth cookie is encrypted and signed with ASP.NET Core Data Protection, expires after 8 hours, is HttpOnly, SameSite=Strict, Secure over HTTPS and scoped to the dashboard path. Changing the username or password invalidates existing sessions.
• Brute-force lockout: 5 failed attempts from the same IP within 15 minutes lock that IP out for 15 minutes (429).
• CSRF protection: every state-changing UI action is POST-only and requires an anti-forgery token; state-changing API calls require Content-Type: application/json.
• Output encoding: all captured data is HTML-encoded before rendering, so payloads such as <script> in a request body are displayed as text.
• Response headers: Content-Security-Policy, X-Frame-Options: DENY, X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer and Cache-Control: no-store on every dashboard response.
• IP allow-list: Security.AllowedIps restricts who can reach the dashboard and API.

What you must configure

1. Enable authentication outside local development and load the password from a secret store.

2. Serve the app over HTTPS, otherwise credentials and the session cookie travel in clear text.

3. Restrict by network with Security.AllowedIps (VPN / office ranges), and call UseForwardedHeaders() before UseHubble() when behind a proxy.

4. Persist Data Protection keys when running more than one instance (or when instances are recycled), so a session cookie issued by one instance is valid on the others:

   builder.Services.AddDataProtection()
       .SetApplicationName("Orders.Api")
       .PersistKeysToFileSystem(new DirectoryInfo("/var/keys/orders-api")); // or Redis, Azure Blob, etc.
   

5. Do not expose Hubble routes through a permissive CORS policy (for example SetIsOriginAllowed(_ => true).AllowCredentials()), which would defeat the API's CSRF protection.

6. Consider AllowDeleteAll = false in shared environments.

Capturing ILogger messages

builder.Logging.AddHubbleLogging(); // uses options.MinimumLogLevel (Information by default)

Messages logged while a request is being processed are linked to it and appear in the Loggers section of the request detail page; messages logged outside a request are stored as standalone entries.

Keep the volume under control with the standard logging filters, scoped to Hubble's provider:

using Gabonet.Hubble.Logging;

builder.Logging.AddHubbleLogging(LogLevel.Information);
builder.Logging.AddFilter<HubbleLoggerProvider>("Microsoft", LogLevel.Warning);
builder.Logging.AddFilter<HubbleLoggerProvider>("System", LogLevel.Warning);

AddHubbleLogging() depends on the services registered by AddHubble(). If you enable Hubble conditionally, register both (and UseHubble()) under the same condition.

Background jobs (Hangfire)

Hubble can show what your background jobs did, without opening the Hangfire dashboard or reading log files on the server. The integration lives in a separate package, so Hangfire is never a dependency of Gabonet.Hubble itself:

dotnet add package Gabonet.Hubble.Hangfire

using Gabonet.Hubble.Extensions;
using Gabonet.Hubble.Hangfire;

builder.Services.AddHubble(builder.Configuration.GetSection("Hubble"));
builder.Logging.AddHubbleLogging();

builder.Services.AddHangfire(config => config.UseRedisStorage(redis)); // any storage
builder.Services.AddHangfireServer();                                  // or app.UseHangfireServer(...)

builder.Services.AddHubbleHangfire(); // after AddHubble(); before or after AddHangfire()

The dashboard gets a Jobs section ({BasePath}/jobs):

• Recurring jobs, read from Hangfire's storage: id, method, queue, cron (with a description such as every 15 minutes), time zone, last execution with its result and duration, next execution with a live countdown (overdue when no server picked it up), status (Active, Error — invalid cron or a method that no longer exists — or Not scheduled) and the result of the last 20 executions.
• On-demand jobs (BackgroundJob.Enqueue, Schedule, continuations), grouped by Type.Method.
• Execution history of each job, filterable by result (succeeded, failed, interrupted, running), with start, duration, retry number, server, and the number of logs, warnings and errors.
• Execution detail: result, error with its stack trace (the real exception, not Hangfire's wrapper), arguments and the ILogger messages written during that execution, with level filter, search and the time of each message since the start.
• Hangfire counters: enqueued, scheduled, processing and failed jobs, servers and queues.

ILogger messages written while a job runs are linked to its execution exactly like messages written during a request are linked to the request: the Logs section shows JOB listen_mailbox instead of an empty URL, and the log detail links to the execution. The link follows the job's asynchronous flow (await, Task.Run), so it works for async jobs too.

How it works:

• AddHubbleHangfire() adds a server filter to the IJobFilterProvider registered in the container, which is the one AddHangfireServer() and app.UseHangfireServer() use. It runs first, so it measures the whole job. If you build BackgroundJobServer yourself with your own FilterProvider, add the filter by hand: GlobalJobFilters.Filters.Add(app.ApplicationServices.GetRequiredService<HubbleHangfireJobFilter>()).
• Each execution is stored in the HubbleJobRuns collection when it starts (so long jobs show as running) and again when it ends. Writes go through an in-memory queue, like the logs: a job never waits for MongoDB.
• Executions follow the same retention as the logs (EnableDataPrune / MaxLogAgeHours, TTL index hubble_job_started) and are deleted by Delete all.
• An execution that is still running in Hubble while Hangfire no longer processes it (the process was restarted or crashed mid-job) is shown as interrupted.

Options (Hubble:Jobs in appsettings.json):

Option	Default	Description
Jobs.Enabled	true	Set to false to turn the module off without touching code: AddHubbleHangfire() then registers nothing.
Jobs.CaptureArguments	true	Store each execution's arguments as JSON, masked with Security.MaskBodyProperties.

Add Hangfire's own dashboard path to IgnorePaths (for example "/hangfire" or "/jobs"): its polling would otherwise flood the request list.

Other schedulers. The tracking is not tied to Hangfire. AddHubbleJobs() enables the Jobs section and HubbleJobTracker, which any background code can use:

public class InvoiceSync(HubbleJobTracker tracker, ILogger<InvoiceSync> logger) : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using var run = tracker.Start(new HubbleJobStart { JobName = "InvoiceSync.Run", Source = "Worker" });
        try
        {
            logger.LogInformation("Synchronizing invoices"); // linked to this execution
            await SyncAsync(stoppingToken);
        }
        catch (Exception ex)
        {
            run.Fail(ex);
            throw;
        }
    }
}

A sample application with Hangfire 1.7 in memory is in examples/Gabonet.Hubble.HangfireSample.

Capturing database queries

Entity Framework Core

builder.Services.AddDbContext<AppDbContext>((serviceProvider, options) =>
{
    var httpContextAccessor = serviceProvider.GetRequiredService<IHttpContextAccessor>();
    options
        .UseSqlServer(builder.Configuration.GetConnectionString("Default"))
        .AddHubbleInterceptor(httpContextAccessor, "AppDatabase");
});

ADO.NET

Wrap the connection so every command is captured:

using var connection = dbContext.GetTrackedConnection(httpContextAccessor, "AppDatabase");
await connection.OpenAsync();

using var command = connection.CreateCommand();
command.CommandText = "[dbo].[GetOrderById]";
command.CommandType = CommandType.StoredProcedure;
command.Parameters.Add(new SqlParameter("@Id", id));

using var reader = await command.ExecuteReaderAsync();

Or capture a single command explicitly before executing it:

command.CaptureAdoNetCommand(httpContextAccessor, "AppDatabase");

MongoDB

Subscribe to the driver's command events in your own MongoClient:

var settings = MongoClientSettings.FromConnectionString(connectionString);
settings.ClusterConfigurator = cb =>
{
    cb.Subscribe<CommandStartedEvent>(e =>
    {
        var httpContext = httpContextAccessor.HttpContext;
        if (httpContext == null) return;

        httpContext.AddDatabaseQuery(new DatabaseQueryLog(
            databaseType: "MongoDB",
            databaseName: databaseName,
            query: e.Command.ToJson(),
            parameters: null,
            callerMethod: MongoDbExtensions.GetCallerMethod(),
            tableName: e.Command.GetCollectionName(),
            operationType: e.CommandName));
    });
};
var client = new MongoClient(settings);

SQL parameters and MongoDB commands are stored as executed, without masking. Avoid capturing queries that carry secrets, or exclude those endpoints with IgnorePaths.

Tagging endpoints

Tags group requests by what they do (an integration, a module, a team) so you can filter the dashboard by them and, in production, capture only the ones you care about. A request can get tags from three places, and they add up:

using Gabonet.Hubble;
using Gabonet.Hubble.Extensions;

// 1. Controllers and actions
[HubbleTag("SAP", "Billing")]
public class InvoicesController : ControllerBase
{
    [HttpPost, HubbleTag("Payments")]
    public IActionResult Pay() => Ok();
}

// 2. Minimal APIs and route groups
app.MapPost("/api/sap/orders", SendOrder).WithHubbleTags("SAP");

// 3. At runtime, from anywhere that has the HttpContext (for example, right before calling an external system)
HttpContext.AddHubbleTag("SAP");

Or by path, without touching the code:

options.TagPaths["SAP"] = new List<string> { "/api/sap", "/api/*/sap/*" };

"Hubble": { "TagPaths": { "SAP": [ "/api/sap", "/api/*/sap/*" ], "Auth": [ "/api/auth" ] } }

A path without * matches by segment (/api/sap includes /api/sap/orders but not /api/sapphire); with * it is a wildcard pattern. Matching is case-insensitive.

In the dashboard, the filter panel lists every tag of the project — the ones declared with [HubbleTag], WithHubbleTags and TagPaths, plus the ones found in stored requests — as a multi-select: a request is shown when it has at least one of the selected tags. Tags are also shown on each row (click one to filter by it) and can be typed in the search box as tag:SAP.

Searching

The search box of the request list looks in several fields at once. Terms separated by a space, a comma or + must all match, each one in any of the searchable fields, so these three searches are equivalent and find the logins made by p2payservice:

auth/login + p2payservice
auth/login, p2payservice
auth/login p2payservice

Use quotes for a literal phrase ("System Account") and tag:Name to require a tag. Matching is case-insensitive.

Your application decides which fields are searched with SearchFields (a flags enum, also bindable from appsettings.json as "Url, RequestBody"):

Flag	Field
Url	Request path
QueryString	Query string
RequestBody	Request body
ResponseBody	Response body
ErrorMessage	Error message
Tags	Tags
RequestHeaders	Request headers (not in the default)
Controller	Controller and action names (not in the default)
IpAddress	Client IP (not in the default)

using Gabonet.Hubble.Models;

options.SearchFields = HubbleSearchFields.Url | HubbleSearchFields.RequestBody | HubbleSearchFields.Tags;

The default is HubbleSearchFields.Default (Url | QueryString | RequestBody | ResponseBody | ErrorMessage | Tags). Searching bodies is an unindexed scan of the stored documents: on very large collections, narrow SearchFields or keep retention short. Masked values (*****) cannot be searched.

Files in requests and responses

Uploads and downloads are no longer stored as text:

• A multipart form is stored as a JSON object with its fields (masked by field name with the same Security.Mask* settings) and each file is kept separately.
• A binary body (PDF, image, ZIP, application/octet-stream, ...) in a request or a response is kept as a file. Compressed responses (gzip, br, deflate) are decompressed before being stored.

In the detail page and in the quick preview, each file shows its name, type and size with two actions: Ver opens PDFs, images and plain text in a new browser tab, and Descargar downloads it. Any other type is always served as a download, so a captured HTML or SVG file can never run in the dashboard's origin.

Option	Default	Description
CaptureFileContents	true	Store file contents. With false only the metadata (name, type, size) is stored.
MaxFileCaptureBytes	5242880 (5 MB)	Larger files keep only their metadata. Maximum 12 MB (a MongoDB document cannot exceed 16 MB).

Bodies larger than 32 MB are never read into memory for capture; only their metadata is recorded. Entries captured by earlier versions (where files were saved as text) are still readable: multipart forms are rebuilt from the stored text (fields and file names) and binary text is loaded only on demand instead of being dumped into the page.

File contents are stored inside the log document, so they count towards your MongoDB storage and follow the same retention. Set CaptureFileContents = false if your uploads are sensitive or large.

Selective capture (production)

By default Hubble captures every request. CaptureOnly turns that into an allow-list: as soon as one condition is configured, a request is stored only if it meets at least one of them.

// Production: keep only failures and everything sent to SAP
options.CaptureOnly.MinimumStatusCode = 500;
options.CaptureOnly.Tags.Add("SAP");

"Hubble": {
  "CaptureOnly": { "MinimumStatusCode": 500, "Tags": [ "SAP" ] },
  "MinimumLogLevel": "Warning"
}

Condition	Captures
MinimumStatusCode	Requests whose status code is that or higher (500 for server errors, 400 to include client errors), and requests that end with an unhandled exception.
StatusCodes	Requests with one of these exact status codes.
Tags	Requests with one of these tags. "*" means any tagged request.
MinimumDurationMs	Requests that take that long or longer.

ILogger messages written during a request follow it: they are stored with the request when it is captured and discarded when it is not. Messages written outside a request (background services, startup) are still captured according to MinimumLogLevel, so raise it in production. Being configuration, the rules can differ per environment (appsettings.Production.json).

Masking sensitive data

Masking applies to JSON request and response bodies, to the fields of multipart forms and to request headers:

• Property matching is case-insensitive and recursive (nested objects and arrays).
• MaskBodyProperties applies to both directions; MaskRequestBodyProperties and MaskResponseBodyProperties add direction-specific properties.
• Masked values are replaced with ***** before anything is written to MongoDB.

Other bodies (URL-encoded form posts, plain text) and query strings are not masked, and the contents of uploaded files are stored as they are (see Files in requests and responses); see Known limitations.

Masking your own logs

HubbleMaskingHelper reuses the same configuration:

using Gabonet.Hubble.Utilities;

_logger.LogInformation("Creating user: {User}", HubbleMaskingHelper.SerializeMasked(userDto));
// Creating user: {"Name":"John Doe","Email":"john@example.com","Password":"*****"}

// Extra properties for a single call, optional indentation
var json = HubbleMaskingHelper.SerializeMasked(order, new List<string> { "iban" }, indent: true);

// Already-serialized JSON
var masked = HubbleMaskingHelper.MaskJson(rawJson);

How capture works (and what happens if MongoDB is down)

Hubble never makes your application wait for MongoDB:

1. When a request arrives, Hubble creates the log entry in memory (its id is generated locally), so ILogger messages can be linked to it immediately.
2. When the request finishes, the entry is placed in a bounded in-memory queue (up to 5,000 entries). ILogger messages go to the same queue.
3. A background service drains the queue and inserts the entries in batches of up to 200.

If MongoDB is slow or unavailable:

• Your endpoints and application startup are unaffected.
• The first failed write prints a [Hubble] warning to the console (later failures only with EnableDiagnostics), and a recovery message is printed when MongoDB comes back.
• While MongoDB is down, new entries are dropped once the queue is full, instead of growing memory without limit.
• On shutdown Hubble tries to flush pending entries for up to 5 seconds (skipped if MongoDB is already known to be down).
• The dashboard itself needs MongoDB; it reports an error after the server selection timeout (5 seconds by default).

Entries appear in the dashboard a few milliseconds after the request completes.

Data retention and MongoDB setup

Give Hubble its own database and a dedicated MongoDB user with readWrite on that database only. Hubble creates its own MongoClient from ConnectionString and does not register it in dependency injection, so it is fully independent from any IMongoClient your application uses. Hubble uses three collections: HubbleLogs, HubbleStats and HubbleConfig.

Indexes and retention are managed for you

At startup a background service prepares the HubbleLogs collection, without delaying your application:

Index	Keys	Purpose
hubble_timestamp	{ timestamp: 1 }	Sorts the dashboard list. With EnableDataPrune = true it is a TTL index: MongoDB deletes logs older than MaxLogAgeHours.
hubble_related_request	{ relatedRequestId: 1 } (partial)	Loads the ILogger entries shown in a request's detail page.
hubble_tags	{ tags: 1 }	Filters the dashboard by tag.

• Retention is enforced by MongoDB itself (its TTL monitor runs about once a minute), so there is no prune job in your application and it keeps working even if the app is stopped.
• Changing MaxLogAgeHours or toggling EnableDataPrune updates the index on the next startup. The index is dropped and recreated, which only needs the readWrite role.
• Indexes you created by hand are respected: an existing index on the same keys is reused. The only exception is a TTL on { timestamp: 1 } that conflicts with EnableDataPrune = true, which is replaced by Hubble's.
• If MongoDB is unreachable, the service retries every 30 seconds. The result is shown under Índices de MongoDB on the dashboard's /config page.
• The dashboard's manual prune (POST /api/prune) is still available and uses the configured MaxLogAgeHours.

Healthy implementation checklist

• UseHubble() is registered early, after UseForwardedHeaders() and before UseRouting().
• Health checks, metrics, Swagger, file downloads, SSE/streaming and other high-volume endpoints are in IgnorePaths.
• RequireAuthentication is on outside local development, with the password loaded from a secret store.
• The app is served over HTTPS.
• Security.AllowedIps limits the dashboard to trusted networks.
• Data Protection keys are persisted and shared when running multiple instances.
• Masking lists cover the sensitive fields of your domain (the defaults are only a starting point).
• Retention is enabled (EnableDataPrune + MaxLogAgeHours) and /config reports the MongoDB indexes as created.
• Hubble uses its own MongoDB database and least-privilege credentials.
• ILogger capture is filtered so framework noise does not flood the dashboard.
• With Hangfire, AddHubbleHangfire() is registered and Hangfire's dashboard path is in IgnorePaths.

Web UI and REST API

Route (relative to BasePath)	Method	Description
/	GET	HTTP requests, with filters (method, statusGroup, tag — repeatable —, url for the search text) and pagination
/logs	GET	ILogger messages (filter by level, search in category and message)
/errors	GET	Failed requests (5xx or unhandled exception) and logs of level Error or higher
/queries	GET	Requests that executed database queries
/stats	GET	Traffic, error and response-time statistics (?range=1h, 24h, 7d or all)
/detail/{id}	GET	Request detail, related ILogger messages and queries
/preview/{id}	GET	HTML fragment with the request and response bodies, used by the list's quick preview
/file/{id}/{index}	GET	A captured file. PDFs, images and plain text are served inline; anything else, or with ?download=1, as a download
/jobs	GET	Recurring and on-demand jobs, with their last and next execution (Jobs module)
/jobs/runs	GET	Execution history (job = recurring id or Type.Method, status = succeeded, failed, canceled or running, page)
/jobs/run/{id}	GET	Execution detail with its error, arguments and ILogger messages
/config	GET	Storage statistics and effective configuration
/delete-all	POST	Delete every log (anti-forgery token required; honours AllowDeleteAll)
/login, /logout	POST / GET	Session management when authentication is enabled
/api/...	GET / POST / DELETE	JSON API

The JSON API (logs, details, jobs, configuration, prune, statistics) is documented in docs/API_ENDPOINTS.md. API clients authenticate with HTTP Basic:

curl -u admin:password "https://your-app/hubble/api/logs?pageSize=20"
curl -u admin:password -X POST -H "Content-Type: application/json" "https://your-app/hubble/api/prune"
curl -u admin:password "https://your-app/hubble/api/jobs/runs?job=listen_mailbox&status=failed"

pageSize is capped at 200.

Building from source

Prerequisites

• .NET SDK 8.0 or later. Running the tests for every target also needs the .NET 6 and .NET 8 runtimes.
• A MongoDB instance for manual testing (for example docker run -d -p 27017:27017 mongo:7).

Build and pack

git clone https://github.com/GaboIV/Gabonet.Hubble.git
cd Gabonet.Hubble
dotnet build src/src.csproj -c Release

Both projects have GeneratePackageOnBuild enabled, so a Release build produces each .nupkg under its bin/Release folder.

To produce the packages for publishing, use the pack script. It deletes bin and obj, builds both projects in Release, copies the packages to nupkg/ and verifies them (assembly version of every DLL for net6.0, net7.0 and net8.0, the Jobs types in Gabonet.Hubble.dll, and that Gabonet.Hubble.Hangfire references and depends on the same version of Gabonet.Hubble):

pwsh build/pack.ps1
pwsh build/verify-packages.ps1 -Version 0.4.2 -PackageDirectory nupkg   # verify existing packages only

Do not use dotnet pack on its own: with GeneratePackageOnBuild, NuGet does not build before packing and packs whatever is in bin. That is how Gabonet.Hubble 0.4.1 shipped the 0.4.0 DLL. Both projects import build/Hubble.Pack.targets, which now stops the pack when a DLL in bin does not have the project's version.

To try the packages (not the projects) in the Hangfire sample: dotnet run --project examples/Gabonet.Hubble.HangfireSample -p:UseHubblePackages=true -p:HubblePackageVersion=0.4.2.

Running the tests

dotnet test

The suite runs without any external service; the Hangfire tests use Hangfire 1.7.24 with in-memory storage. The integration tests for indexes, TTL, the full capture pipeline and the job execution store need a real MongoDB and are reported as skipped unless the HUBBLE_TEST_MONGODB environment variable points to one. Each of them uses its own temporary database and drops it afterwards.

docker run -d --name hubble-test-mongo -p 27017:27017 mongo:7
HUBBLE_TEST_MONGODB=mongodb://localhost:27017 dotnet test

Testing a local build in another application

dotnet add package Gabonet.Hubble --source /path/to/Gabonet.Hubble/artifacts

Or reference the project directly while developing:

<ProjectReference Include="..\Gabonet.Hubble\src\src.csproj" />

Repository layout

Path	Content
src/Middleware	Capture middleware and HubbleOptions
src/BackgroundServices	Background log writer and storage initializer (indexes, TTL retention)
src/UI	Dashboard middleware, HTML rendering and API models
src/Security	IP allow-list, login throttling, session tokens, HTML encoding
src/Services	Log queue, MongoDB-backed log and statistics services
src/Logging	ILogger provider
src/Extensions	Service registration, EF Core / ADO.NET capture, MongoDB helpers
src/Utilities	HubbleMaskingHelper
src/Jobs	Jobs module: HubbleJobTracker, execution model, scheduler and store contracts
integrations/Gabonet.Hubble.Hangfire	Gabonet.Hubble.Hangfire package: Hangfire server filter and recurring jobs catalog
build/	pack.ps1 (clean build and verified packages), verify-packages.ps1 and the pack guard Hubble.Pack.targets
docs/	API reference and masking guide
examples/	Sample Program.cs, appsettings.json, authentication guide and a runnable Hangfire sample
tests/Gabonet.Hubble.Tests	xUnit test suite (security, capture, resilience, options, jobs, Hangfire, MongoDB integration)

Troubleshooting

Symptom	Cause and fix
Dashboard returns 403 Access denied: IP not allowed	Your IP is not in Security.AllowedIps. Behind a proxy, configure UseForwardedHeaders() before UseHubble().
Login returns 429	Too many failed attempts from your IP. Wait 15 minutes.
"Invalid or missing anti-forgery token"	The page was open too long or cookies are blocked. Reload the page and retry.
Logged out after a deploy or on another instance	Data Protection keys are not persisted/shared. See Securing the dashboard.
App fails at startup with "RequireAuthentication está activado…"	Authentication is enabled with an empty username or password. Provide both.
API returns 415	POST requests must send Content-Type: application/json.
No ILogger entries under a request	Check AddHubbleLogging() is registered, the level/filters allow the message, and that it was logged during the request.
Dashboard pages are slow	Check on /config that the MongoDB indexes are created, and enable retention so the collection does not grow forever.
[Hubble] No se pudieron preparar los índices de MongoDB in the console	MongoDB was unreachable at startup or the user lacks readWrite. Hubble retries every 30 seconds; the app is not affected.
The Jobs section says the module is not enabled	Call AddHubbleHangfire() (package Gabonet.Hubble.Hangfire) after AddHubble(), and check Hubble:Jobs:Enabled is not false.
Job executions appear, but without logs	The job's logger category starts with Gabonet.Hubble (Hubble ignores its own categories), the messages are below MinimumLogLevel, or they were written from a thread that does not flow from the job (a Thread or timer started elsewhere).
Recurring jobs are listed but no execution is recorded	The Hangfire server was built with its own BackgroundJobServerOptions.FilterProvider. Add HubbleHangfireJobFilter to GlobalJobFilters.Filters (see Background jobs).
An execution shows as interrupted	Hubble recorded its start but not its end, and Hangfire no longer processes it: the process stopped mid-job. Hangfire retries it as a new execution.
[Hubble] No se pudieron guardar N logs en MongoDB in the console	MongoDB is unreachable from the app. Your application keeps working; logs produced meanwhile are dropped. Check the connection string and network.

Known limitations

These are tracked for upcoming releases. Plan around them today:

• Response buffering: every captured response is buffered in memory. Add streaming endpoints, server-sent events and large downloads to IgnorePaths.
• ILogger provider cost: each captured message walks the stack trace to find its source. Use filters to limit volume.
• Dashboard configuration page is read-only: the effective configuration comes from code / appsettings.json.
• Masking scope: only JSON bodies, multipart form fields and request headers are masked; query strings, URL-encoded form posts, uploaded file contents, SQL parameters and ILogger messages are stored as-is.
• Selective capture and ILogger: CaptureOnly decides per request. Messages logged outside Hubble's middleware (hosting start/finish messages, background services) are not tied to a request and are governed by MinimumLogLevel and the logging filters.
• Tag filter and ILogger messages: tags belong to requests; the ILogger section is not filtered by tag.
• Jobs and database queries: the EF Core / ADO.NET interceptors attach queries to the current HTTP request, so queries run by a background job are not captured.
• Jobs and Hangfire's own messages: messages Hangfire writes after the job has finished (for example the AutomaticRetry warning announcing a retry) are stored as standalone logs, not linked to the execution. The execution itself shows the error and the retry number.

Upgrade notes

0.4.2

• Packaging fix, no code changes. Gabonet.Hubble 0.4.1 on NuGet contains the 0.4.0 DLL (no Jobs module), so Gabonet.Hubble.Hangfire 0.4.1 fails at startup with FileLoadException: Could not load file or assembly 'Gabonet.Hubble, Version=0.4.1.0'. Use 0.4.2 for both packages. See CHANGELOG_v0.4.2.md.

0.4.1

• New optional package Gabonet.Hubble.Hangfire and Jobs module. Nothing changes until you call AddHubbleHangfire() (or AddHubbleJobs()).
• GeneralLog has two new fields, JobRunId and JobName, only set on ILogger messages written during a job execution. Existing documents are read without changes.
• New option Jobs (Enabled, CaptureArguments) in HubbleOptions.
• HubbleController's constructor has three new optional parameters (IHubbleJobStore, IHubbleJobCatalog, HubbleJobTracker), resolved by dependency injection.
• With the Jobs module enabled Hubble creates a HubbleJobRuns collection with two indexes (hubble_job_key, hubble_job_started) and one more index on HubbleLogs (hubble_job_run). Delete all also deletes the job executions.

0.4.0

• GeneralLog has new fields: Tags, Attachments, RequestContentType and ResponseContentType. Existing documents are read without changes.
• Multipart forms are now stored as a JSON object with their fields (RequestData) plus one attachment per file, and binary bodies are stored as attachments with an empty text body. Previously both were saved as (corrupted) text.
• The dashboard search now looks in the request and response bodies, error message and tags as well as the URL (see SearchFields), and treats spaces, commas and + as term separators.
• IHubbleService has two new members, GetAttachmentAsync and GetTagsAsync. Custom implementations must add them.
• GetLogByIdAsync, QueryLogsAsync and GetRelatedLogsAsync no longer return file contents (HubbleAttachment.Data is null); use GetAttachmentAsync.
• Hubble creates one more index at startup, hubble_tags.
• HubbleController's constructor has a new optional parameter (IEnumerable<EndpointDataSource>), resolved by dependency injection.
• Request bodies are now read completely before capture (a single read could previously truncate large bodies), and bodies over 32 MB are not captured.

0.3.0

This release hardens the dashboard and contains breaking changes:

• Security.AllowedIps now applies only to the dashboard and API. Previously it was evaluated for every request and could block the whole application.
• When configuring from appsettings.json, Security.AllowedIps no longer defaults to 127.0.0.1; an empty list allows any IP.
• Security.MaskRequestBodyProperties and Security.MaskResponseBodyProperties are now read from appsettings.json.
• GET /api/prune and GET /api/recalculate-stats were replaced by POST (with Content-Type: application/json). Every POST to the API now requires that content type.
• /delete-all, /run-prune, /recalculate-stats and the /save-* routes only accept POST with an anti-forgery token.
• Session cookies issued by previous versions are no longer accepted; users must log in again.
• The application fails at startup if RequireAuthentication is enabled with an empty username or password, or if BasePath is empty or /.
• The package now uses the shared ASP.NET Core framework (Microsoft.AspNetCore.App) instead of the deprecated Microsoft.AspNetCore.* 2.2 packages, and registers Data Protection and Antiforgery services.
• HubbleController's constructor now also requires IAntiforgery and IHttpContextAccessor (both registered by AddHubble()).
• The logout button is only shown when authentication is enabled.
• Captured logs are written asynchronously in batches by a hosted service instead of inline in each request. Requests no longer wait for MongoDB, and logs are dropped (not queued forever) while MongoDB is unavailable.
• HubbleMiddleware now takes HubbleOptions and HubbleLogQueue; HubbleLoggerProvider now takes HubbleLogQueue, IHttpContextAccessor and HubbleOptions (the constructors based on IHubbleService were removed). Both are created for you by UseHubble() / AddHubbleLogging().
• Retention is now enforced by a MongoDB TTL index that Hubble creates and maintains. HubbleDataPruneManager and DataPruneService were removed: nothing deletes logs from inside your application anymore, and DataPruneIntervalHours is ignored.
• Hubble now creates its indexes at startup (hubble_timestamp, hubble_related_request). Indexes created by hand from the previous README are reused, except a { timestamp: 1 } TTL with a different age when EnableDataPrune is on, which is replaced.
• The application fails at startup if EnableDataPrune is on and MaxLogAgeHours is outside 1–596,523.
• The /config page shows the effective retention settings and the index status instead of the values stored in MongoDB on first run, and the manual prune uses the configured MaxLogAgeHours.
• HubbleStatsService no longer queries MongoDB in its constructor, and neither the prune service nor the log writer delays application startup.
• Hubble's MongoDB client defaults to a 5-second server selection timeout unless the connection string sets serverSelectionTimeoutMS.
• AddHubble() no longer registers an IMongoClient in the service collection. Hubble keeps its own private client, so it can no longer replace the client your application registers for its own data (previously, whichever registration came last won). If your code resolved IMongoClient from DI only because Hubble registered it, register your own client explicitly.
• CaptureHttpRequests = false now fully disables HTTP capture, and request logs now include the client IP address.
• Log categories starting with Gabonet.Hubble are no longer captured by the ILogger provider.
• Single options class. HubbleOptions (namespace Gabonet.Hubble.Middleware) is now the only configuration type and includes ConnectionString, DatabaseName and MinimumLogLevel. HubbleConfiguration, HubbleAuthConfiguration and the duplicate Gabonet.Hubble.Models.SecurityConfiguration were removed. Code using AddHubble(options => { ... }) keeps compiling unchanged.
• New AddHubble(IConfigurationSection, Action<HubbleOptions>?) overload based on the standard configuration binder; every option can now be set from appsettings.json and from code (previously some options were only available in one of them).
• AddHubble(connectionString, databaseName, ...) and AddHubble(IConfiguration, connectionString, databaseName, sectionName) still work but are marked [Obsolete]. The former now also registers the statistics service, so the /config page works with it.
• All registrations validate the options and fail at startup with every error listed. A TimeZoneId that does not exist on the machine is now an error instead of silently falling back to UTC.
• CaptureLoggerMessages (default now true) and MinimumLogLevel are honoured. AddHubbleLogging() without arguments uses MinimumLogLevel; passing a level still takes precedence.
• The /config page shows the effective configuration (service, database, time zone, capture settings, ignored paths) instead of values stored in MongoDB on first run or read from undocumented HUBBLE_* environment variables.

License

Released under the MIT License.

Contributing

Issues and pull requests are welcome.