Sprint.Shared.Logs 1.3.2

Shared.Logs

High-performance structured logging library designed for the Sprint ecosystem (.NET 10+). Provides two independent logging pipelines — Application Event Logging (via Serilog Sink) and HTTP Request/Response Logging (via Middleware) — both using asynchronous batch writes to MySQL with automatic table partitioning.

Key Features

• Non-blocking Architecture: All log entries are queued in-memory (ConcurrentQueue) and flushed to MySQL by a background worker — zero impact on request latency.
• Requires .NET 10.0+
• Auto MySQL Partitioning: Tables are auto-created and partitioned by a configurable day interval (default: every 7 days, 365 days ahead). Partition boundaries continue seamlessly from the last existing partition on each deploy — no gaps, no data loss.
• Dual Logging Pipelines:
  • AddAppLogging — Captures structured Serilog events (level, message, exception, trace ID, metadata) into logs_app table.
  • AddHttpLogging — Captures full HTTP pairs (method, URL, headers, body, status code, duration) into logs_http table.
• Configurable Batch Flush: Flush is triggered when batch reaches BatchSize OR when BatchTimeoutSeconds elapses — whichever comes first.
• Path Filtering: HTTP logging supports IgnorePaths to skip health checks, metrics endpoints, etc.
• App Log Level Filtering: AppLogMinimumLevel controls the minimum Serilog level stored in MySQL (e.g. Warning to skip Information/Debug entirely).
• Outbound HTTP Tracing: HttpLoggingHandler (DelegatingHandler) logs outgoing HTTP client calls to a dedicated MySQL table (default logs_http_partner) via IHttpLogQueue, with table name configurable per deployment.
• Multi-table Support: MySqlHttpLogRepository accepts an optional tableName parameter — register separate pipelines for inbound and outbound logs without any code duplication.
• Reference Header Indexing: Configure SharedLog:ReferenceHeaders to extract business-key headers (e.g. x-order-id, x-log-key) into a separate logs_reference table, enabling fast reverse lookup from a business key back to the originating TraceId.

Installation & Setup

1. Register Services in Program.cs

App Event Logging (Serilog → MySQL)

using Shared.Logs.Extensions;

// Option 1: Automatic from Configuration
builder.AddAppLogging(builder.Configuration);

// Option 2: Manual setup via Action
builder.AddAppLogging(options => {
    options.ConnectionString = "Server=localhost;Database=logs_db;...";
    options.ServiceName = "Order-Core-API";
    options.BatchSize = 200;
    options.BatchTimeoutSeconds = 10;
});

HTTP Request/Response Logging

// Option 1: Automatic from Configuration
builder.AddHttpLogging(builder.Configuration);

// Option 2: Manual setup via Action
builder.AddHttpLogging(options => {
    options.ConnectionString = "Server=localhost;Database=logs_db;...";
    options.ServiceName = "Order-Core-API";
    options.IgnorePaths = new List<string> { "/health", "/metrics" };
});

2. Enable Middleware in Program.cs

var app = builder.Build();

// Enable HTTP logging middleware (must be registered early in the pipeline)
app.UseHttpLogging("Order-Core-API");

app.Run();

3. Configuration Template (appsettings.json)

{
  "ConnectionStrings": {
    "LogEvent":   "Server=localhost;Port=3306;Database=sprint_logs;Uid=root;Pwd=secret;",
    "LogRequest": "Server=localhost;Port=3306;Database=sprint_logs;Uid=root;Pwd=secret;"
  },
  "SharedLog": {
    "ServiceName":          "Sprint-OMS-Order-API",
    "BatchSize":            100,
    "BatchTimeoutSeconds":  5,
    "AppLogMinimumLevel":   "Warning",
    "PartitionIntervalDays":  7,
    "PartitionLookAheadDays": 365,
    "IgnorePaths": [
      "/health",
      "/favicon.ico",
      "/metrics"
    ],
    "ReferenceHeaders": [
      "x-log-key",
      "x-order-id"
    ]
  }
}

App Log Minimum Level

Control which Serilog levels get persisted to MySQL via SharedLog:AppLogMinimumLevel. Accepts any Serilog LogEventLevel value:

Serilog console output is unaffected — this filter applies only to MySQL storage.

Outbound HTTP Client Tracing

HttpLoggingHandler is a DelegatingHandler that logs every outgoing HTTP call to MySQL. It requires an IHttpLogQueue instance — register a keyed "outbound" queue pointing to a separate table:

// In your integration DI registration:
var connStr = configuration.GetConnectionString("LogRequest") ?? string.Empty;
var serviceName = configuration["SharedLog:ServiceName"] ?? "unknown";
var outboundTable = configuration["SharedLog:OutboundTableName"] ?? "logs_http_partner";

services.AddKeyedSingleton<IHttpLogQueue, HttpLogQueueService>("outbound");
services.AddHostedService(sp => new HttpLogProcessingWorker(
    sp.GetRequiredKeyedService<IHttpLogQueue>("outbound"),
    new MySqlHttpLogRepository(new HttpLogOptions { ConnectionString = connStr, ServiceName = serviceName }, outboundTable),
    sp.GetRequiredService<ILogger<HttpLogProcessingWorker>>(),
    sp.GetRequiredService<HttpLogOptions>()
));

services.AddTransient<HttpLoggingHandler>(sp => new HttpLoggingHandler(
    sp.GetRequiredService<ILogger<HttpLoggingHandler>>(),
    sp.GetRequiredKeyedService<IHttpLogQueue>("outbound"),
    sp.GetRequiredService<HttpLogOptions>(),
    sp.GetRequiredService<IHttpContextAccessor>()
));

// Attach to Refit/HttpClient:
services.AddRefitClient<IMyClient>()
    .AddHttpMessageHandler<HttpLoggingHandler>();

Outbound rows are distinguishable by IpAddress IS NULL in the logs_http_partner table.

Config (appsettings.json):

"SharedLog": {
  "OutboundTableName": "logs_http_partner"
}

MySQL Table Schema

logs_app — Application Event Log

logs_http / logs_http_partner — HTTP Request/Response Log

logs_reference — Request Header Reference Index

ใช้สำหรับ trace ย้อนกลับจาก business key (เช่น x-order-id: 12345) ไปยัง trace_id ของ HTTP request ที่ส่ง header นั้นเข้ามา ตารางจะถูกสร้างและ partition โดยอัตโนมัติเช่นเดียวกับตารางอื่น เมื่อ SharedLog:ReferenceHeaders ถูกกำหนดอย่างน้อยหนึ่งค่า

มี composite index (reference_key, reference_value) สำหรับ reverse lookup:

-- หา trace_id ทั้งหมดที่ผูกกับ order 12345
SELECT trace_id FROM logs_reference
WHERE reference_key = 'x-order-id' AND reference_value = '12345';

-- join กลับไปดู request log
SELECT h.*
FROM logs_http h
JOIN logs_reference r ON r.trace_id = h.trace_id
WHERE r.reference_key = 'x-order-id' AND r.reference_value = '12345';

ถ้า request ไม่มี header ที่กำหนดใน ReferenceHeaders → ไม่บันทึก row ถ้า ReferenceHeaders ว่าง → ไม่สร้างตาราง ไม่เปิด worker (no-op)

All tables are auto-partitioned by range (PARTITION BY RANGE (TO_DAYS(timestamp))). Partition size and look-ahead window are controlled by PartitionIntervalDays (default: 7) and PartitionLookAheadDays (default: 365). On each deploy the library continues partitions from the last existing boundary — no gap, no data loss.

Resilience Design

• Zero-loss on insert failure: MySQL errors are caught silently — a failed batch does not crash the worker or the application.
• Self-sealing loop: The background worker catches all unhandled exceptions, logs them, and retries after 1 second.
• Graceful shutdown: When Kubernetes sends SIGTERM, both workers drain all remaining queue items to MySQL before the process exits. No log entries are lost during rolling deploys or scale-down events.
• Manual flush for batch jobs: For CronJobs or short-lived processes that exit immediately after finishing work, call FlushAsync() explicitly before the process exits:

// At the end of a batch job
await httpLogQueue.FlushAsync();
await appLogQueue.FlushAsync();

© 2026 Sprint-OMS Development Team