Elven.Observability.Hosting 0.1.13

Elven Observability .NET

Production-grade OpenTelemetry bootstrap for modern .NET services on the Elven Observability LGTM stack: Loki, Grafana, Tempo, and Mimir.

This distribution wraps the official OpenTelemetry .NET SDK and OpenTelemetry .NET AutoInstrumentation. It does not replace OpenTelemetry. Elven mode always configures traces, metrics, and logs.

Architecture

Applications send OTLP to a Collector deployed in the customer environment. The customer-side Collector then forwards telemetry to the Elven Observability backends.

.NET app
  -> OTLP gRPC or HTTP/protobuf
  -> customer-side OpenTelemetry Collector
  -> Elven Observability backend
  -> Tempo, Mimir, Loki, Grafana

The package default endpoint is http://localhost:4317, which is the common local/sidecar Collector endpoint. In production, set OTEL_EXPORTER_OTLP_ENDPOINT or ELVEN_OTLP_ENDPOINT to the customer Collector address.

Keep Elven backend credentials in the Collector, not in the application. In the recommended Collector path, the app only needs to know where the customer-side Collector is. The Collector is responsible for adding Authorization: Bearer ... and x-scope-orgid when forwarding to Elven Observability.

Recommended Path

Start with the lowest-friction rollout path, then move down only when the service needs code-level customization.

Docker Zero-Code

FROM mcr.microsoft.com/dotnet/aspnet:10.0

COPY --from=elvenobservability/dotnet-instrumentation:latest /otel /otel

ENV CORECLR_ENABLE_PROFILING=1 \
    CORECLR_PROFILER={918728DD-259F-4A6A-AC2B-B85E1B658318} \
    CORECLR_PROFILER_PATH=/otel/current/OpenTelemetry.AutoInstrumentation.Native.so \
    DOTNET_ADDITIONAL_DEPS=/otel/AdditionalDeps \
    DOTNET_SHARED_STORE=/otel/store \
    DOTNET_STARTUP_HOOKS=/otel/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll \
    OTEL_DOTNET_AUTO_HOME=/otel \
    OTEL_DOTNET_AUTO_PLUGINS="Elven.Observability.AutoInstrumentation.ElvenPlugin, Elven.Observability.AutoInstrumentation" \
    OTEL_TRACES_EXPORTER=otlp \
    OTEL_METRICS_EXPORTER=otlp \
    OTEL_LOGS_EXPORTER=otlp \
    OTEL_EXPORTER_OTLP_PROTOCOL=grpc \
    OTEL_DOTNET_AUTO_SQLCLIENT_SET_DBSTATEMENT_FOR_TEXT=true \
    OTEL_DOTNET_AUTO_ENTITYFRAMEWORKCORE_SET_DBSTATEMENT_FOR_TEXT=true \
    OTEL_DOTNET_AUTO_ORACLEMDA_SET_DBSTATEMENT_FOR_TEXT=true

WORKDIR /app
COPY . /app
ENTRYPOINT ["dotnet", "/app/MyApp.dll"]

Runtime configuration:

docker run --rm \
  -e OTEL_SERVICE_NAME=billing-api \
  -e OTEL_SERVICE_VERSION=1.0.0 \
  -e ELVEN_SERVICE_NAMESPACE=payments \
  -e ELVEN_ENVIRONMENT=production \
  -e OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector.observability.svc.cluster.local:4317 \
  -e OTEL_EXPORTER_OTLP_PROTOCOL=grpc \
  -e OTEL_TRACES_EXPORTER=otlp \
  -e OTEL_METRICS_EXPORTER=otlp \
  -e OTEL_LOGS_EXPORTER=otlp \
  my-app:latest

Database statements are collected by default when the underlying .NET instrumentation supports them. Elven redacts db.statement, db.query.text, and database query parameter attributes to [REDACTED] before export, while keeping safe attributes such as db.query.summary, db.operation.name, db.system.name, duration, status, and errors.

Process Zero-Code

./scripts/install.sh
. "$HOME/.otel-dotnet-auto/instrument.sh"

export OTEL_DOTNET_AUTO_PLUGINS="Elven.Observability.AutoInstrumentation.ElvenPlugin, Elven.Observability.AutoInstrumentation"
export OTEL_SERVICE_NAME=billing-api
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp

dotnet Billing.Api.dll

Programmatic Setup

Install:

dotnet add package Elven.Observability

ASP.NET Core:

var builder = WebApplication.CreateBuilder(args);

builder.AddElvenObservability(options =>
{
    options.ServiceName = "billing-api";
    options.ServiceNamespace = "payments";
    options.Endpoint = new Uri("http://localhost:4317");
    options.SamplingRatio = 1.0;
});

var app = builder.Build();
app.UseElvenObservability();
app.MapElvenObservabilityHealthCheck();
app.Run();

Generic Host / Worker:

var builder = Host.CreateApplicationBuilder(args);
builder.AddElvenObservability();
await builder.Build().RunAsync();

Console / No Host:

await using var observability = ElvenObservability.Initialize(options =>
{
    options.ServiceName = "batch-job";
});

using var activity = ElvenTelemetry.ActivitySource.StartActivity("job.run");
ElvenTelemetry.Meter.CreateCounter<long>("jobs.started").Add(1);
observability.LoggerFactory.CreateLogger("Batch.Job").LogInformation("Job started.");
await observability.ForceFlushAsync();

Custom Application Telemetry

Do not create a second OpenTelemetry TracerProvider or MeterProvider in an app that already uses AddElvenObservability(). A parallel provider can split context, duplicate exporters, and break CoreWCF parentage.

Register application-owned meters and activity sources inside the Elven pipeline:

builder.AddElvenObservability(options =>
{
    options.ServiceName = "dsg-linux";
    options.ServiceNamespace = "dsg";
    options.AdditionalMeters.Add("DN.DSG");
    options.AdditionalSources.Add("DN.DSG");
});

Environment-only equivalent:

ELVEN_ADDITIONAL_METERS=DN.DSG
ELVEN_ADDITIONAL_SOURCES=DN.DSG

Use this when the application emits custom business metrics such as dsg_command_total or custom spans from new ActivitySource("DN.DSG").

WCF And CoreWCF

Elven.Observability includes WCF client and CoreWCF server instrumentation. Use 0.1.13 or newer for CoreWCF inbound spans in production migrations. Version 0.1.13 keeps ASP.NET Core, CoreWCF, SQL, HTTP, business spans, and correlated logs in the same W3C trace.

Use this when a modern .NET service still calls SOAP/WCF dependencies, or when a migrated service exposes SOAP endpoints through CoreWCF. The instrumentation never captures SOAP payloads by default. It captures operation metadata, latency, status/fault information, exceptions, correlation, and W3C propagation.

Captured WCF attributes include:

• rpc.system=wcf
• rpc.service
• rpc.method
• wcf.contract
• wcf.action
• wcf.binding
• wcf.endpoint
• wcf.route / http.route
• url.path
• server.address
• http.response.status_code
• wcf.is_fault
• correlation_id
• error.category and error.type when an error is detected

WCF client:

using Elven.Observability.Instrumentation.Wcf;

var client = new AereoClient(binding, endpoint);
client.AddElvenObservability(
    serviceName: "dsg-servicos-web",
    bindingName: "basicHttp");

var response = await client.DisponibilidadeAereoAsync(request);

ChannelFactory<T>:

using Elven.Observability.Instrumentation.Wcf;

var factory = new ChannelFactory<IAereo>(binding, endpoint);
factory.AddElvenObservability(
    serviceName: "dsg-servicos-web",
    bindingName: "basicHttp");

var channel = factory.CreateChannel();

CoreWCF server:

Install the ASP.NET Core middleware package explicitly in CoreWCF hosts:

dotnet add package Elven.Observability --version 0.1.13
dotnet add package Elven.Observability.AspNetCore --version 0.1.13
dotnet add package Elven.Observability.Instrumentation.CoreWcf --version 0.1.13

using Elven.Observability.Instrumentation.CoreWcf;

app.UseServiceModel(serviceBuilder =>
{
    serviceBuilder
        .AddService<AereoService>()
        .AddElvenObservability<AereoService>(
            serviceName: "dsg-servicos-web",
            bindingName: "basicHttp");

    serviceBuilder.AddServiceEndpoint<AereoService, IAereo>(
        new BasicHttpBinding(),
        "/Aereo.svc");
});

BasicHttpBinding and WSHttpBinding(SecurityMode.None) are supported for CoreWCF server instrumentation.

CoreWCF service exceptions are observed without changing the generated SOAP Fault or marking the error as handled. The WCF span receives error classification plus a standard exception event; message and stack trace are bounded and sanitized by the active redaction policy before export.

UseElvenObservability() includes a hardened SOAP/.svc fallback. If CoreWCF does not call the dispatch message inspector in a specific hosting shape, the middleware creates an exportable Elven.Observability server span named WCF <contract>/<method> around the SOAP request so downstream SQL, HTTP, business spans, and logs are not trace roots. In 0.1.13, that WCF span is always a child of the active ASP.NET Core request activity when one exists. With ForceSampleCoreWcfInboundSpans=true, the Elven sampler records both the ASP.NET Core hosting parent and the WCF operation without replacing the incoming trace. ASP.NET Core creates the parent before route metadata is available, so this explicit opt-in samples every inbound ASP.NET Core hosting activity in that process; it is intended for dedicated CoreWCF hosts. If no ASP.NET Core activity exists, the fallback uses a valid remote traceparent; otherwise it creates a root WCF span. The selected strategy is tagged as elven.corewcf.parent_strategy=local_parent|remote_parent|root. When the inspector runs later in the CoreWCF pipeline, it reuses and enriches the active WCF span instead of creating a duplicate. The fallback uses request metadata such as path, SOAPAction, Content-Type action, traceparent, and correlation headers. It does not read or export SOAP/XML payloads.

Manual fallback for unusual bindings:

using Elven.Observability.Instrumentation.Wcf;

using var span = ElvenWcfManual.StartOperation(
    service: "dsg-servicos-web",
    contract: "IAereo",
    operation: "DisponibilidadeAereo",
    correlationId: correlationId);

Propagation:

• Outbound WCF client injects traceparent, tracestate, baggage, and x-correlation-id into SOAP headers and HTTP headers when the binding exposes HTTP message properties.
• Inbound CoreWCF extracts traceparent, tracestate, baggage, x-correlation-id, and x-request-id from SOAP headers and HTTP headers.
• If an upstream service sends W3C context, the CoreWCF span keeps the same trace id instead of becoming a new root span.

Kill switches:

ELVEN_ENABLE_WCF=false
ELVEN_ENABLE_COREWCF=false
ELVEN_FORCE_SAMPLE_COREWCF_INBOUND_SPANS=false
ELVEN_WCF_CLIENT_ACTIVITY_TIMEOUT_MS=300000

Runtime diagnostics:

Console.WriteLine(ElvenStartupDiagnostics.InstrumentationHookCount);
Console.WriteLine(ElvenStartupDiagnostics.LastInstrumentationHook);
Console.WriteLine(ElvenStartupDiagnostics.InstrumentationFailureCount);
Console.WriteLine(ElvenStartupDiagnostics.LastInstrumentationFailure);

If SQL spans arrive as trace roots and LastInstrumentationHook does not include corewcf.dispatch, the CoreWCF dispatch inspector was not installed.

For fallback checks, LastInstrumentationHook should include:

corewcf.middleware_fallback: <service-name>

For POST/SOAP requests with an existing ASP.NET Core request activity, the active WCF span should normally have:

source=Elven.Observability
elven.corewcf.fallback=true
elven.corewcf.fallback.mode=child_activity
elven.corewcf.parent_strategy=local_parent

The WCF span must have the same trace_id as the ASP.NET Core request span and its parent_span_id must equal the ASP.NET Core span id. If no ASP.NET Core activity exists and a valid upstream traceparent is available, the fallback uses:

elven.corewcf.fallback.mode=remote_parent
elven.corewcf.parent_strategy=remote_parent

Keep ELVEN_FORCE_SAMPLE_COREWCF_INBOUND_SPANS=true in dedicated CoreWCF services so both the ASP.NET Core parent and WCF operation remain exportable under an unsampled remote parent. This setting does not detach the span. Because the ASP.NET Core sampler runs before route discovery, it also records other inbound ASP.NET Core requests in that process.

Business Telemetry

Install:

dotnet add package Elven.Observability

Use business spans and metrics when the service needs journey-level observability:

using Elven.Observability.Business;
using Elven.Observability.Business.AirTravel;

using var op = ElvenAirTravel.StartAvailability(
    provider: "sabre",
    source: "sabre_bfm",
    correlationId: correlationId);

op.SetStatus("success");
ElvenBusinessMetrics.OperationTotal.Add(1, op.Tags(status: "success"));
ElvenBusinessMetrics.OperationDuration.Record(elapsed, op.Tags(status: "success"));

Business metric labels are intentionally low-cardinality: client, business_context, journey, business.operation, provider, source, supplier, product, status, result, and error.category. Reservation ids, PNRs, tickets, sessions, passengers, and documents must stay out of metric labels.

For unmapped supplier/provider errors, keep the raw high-cardinality message out of metric labels. Put a sanitized, bounded detail on the active business/source span and use a short hash when you need top-k grouping:

op.SetSupplierMessage(mensagemSupplier, missing: string.IsNullOrWhiteSpace(mensagemSupplier));
op.SetErrorDetail("Unmapped supplier response");

ElvenBusinessMetrics.ErrorsTotal.Add(1, op.Tags(
    status: "error",
    errorCategory: "provider_error"));

The standard span attributes are:

• elven.business.supplier_message: sanitized supplier response/message, default max 500 chars.
• elven.business.supplier_message_hash: 8-char SHA-256 prefix for grouping repeated messages.
• elven.business.supplier_message_missing: true when the app only knows that the supplier response was missing/indefinite.
• elven.business.error_detail: sanitized human-readable detail for the business failure.

Air travel helpers cover the customer journey end to end:

ElvenAirTravel.StartOpenSession(provider, source, correlationId);
ElvenAirTravel.StartAvailability(provider, source, correlationId);
ElvenAirTravel.StartPricing(provider, source, correlationId);
ElvenAirTravel.StartReservation(provider, source, correlationId);
ElvenAirTravel.StartPnr(provider, source, correlationId);
ElvenAirTravel.StartPayment(provider, source, correlationId);
ElvenAirTravel.StartAntifraud(provider, source, correlationId);
ElvenAirTravel.StartGateway(provider, source, correlationId);
ElvenAirTravel.StartCancellation(provider, source, correlationId);

See Business telemetry and Business telemetry PT-BR.

Environment Variables

Elven supports standard OpenTelemetry variables plus Elven extensions.

Recommended app-to-Collector variables:

• ELVEN_ENABLED=false: global fail-safe kill switch; no providers, exporters, middlewares, or unhandled-exception observer are registered. OTEL_SDK_DISABLED=true has the same effect. Boolean aliases 0/1, no/yes, off/on, and disabled/enabled are accepted. A disabled value cannot be overridden programmatically.
• ELVEN_ENVIRONMENT: mapped to deployment.environment.name
• ELVEN_REGION: mapped to elven.region
• ELVEN_DEBUG: enables verbose internal diagnostics
• ELVEN_REDACTION_MODE=default|none: default keeps PCI/LGPD redaction enabled
• ELVEN_LOG_BODY_MODE=redacted|raw: default redacts log bodies structurally
• ELVEN_MAX_LOG_BODY_CHARS: default 200000
• ELVEN_ALLOWED_RAW_ATTRIBUTES: exact attribute allowlist for controlled troubleshooting
• ELVEN_ADDITIONAL_METERS: comma-separated custom Meter names collected by the Elven MeterProvider, for example DN.DSG
• ELVEN_ADDITIONAL_SOURCES: comma-separated custom ActivitySource names collected by the Elven TracerProvider, for example DN.DSG
• ELVEN_ENABLE_WCF: disables WCF client instrumentation when set to false
• ELVEN_ENABLE_COREWCF: disables CoreWCF server instrumentation when set to false
• ELVEN_FORCE_SAMPLE_COREWCF_INBOUND_SPANS=true|false: default false; set it to true in a dedicated CoreWCF host to record the ASP.NET Core hosting parent and WCF operation while preserving the original W3C trace; because route metadata is unavailable at sampling time, this opt-in records every inbound ASP.NET Core hosting activity in that process
• ELVEN_SUPPRESS_OTLP_HTTP_SPANS=true|false: default true; compatibility name that now suppresses both HttpClient and gRPC client spans for calls to the configured OTLP Collector endpoint so log/metric/trace exporter traffic does not pollute Tempo
• ELVEN_SUPPRESS_OTLP_EXPORTER_SPANS=true|false: alias for the same exporter self-span suppression with a clearer name
• ELVEN_WCF_CLIENT_ACTIVITY_TIMEOUT_MS: marks long-lived WCF client spans as timeout/error when a reply never closes the span before the configured limit

Direct-to-Elven variables, only when the application exports straight to an Elven OTLP gateway instead of a customer-side Collector:

• ELVEN_TENANT_ID: mapped to x-scope-orgid
• ELVEN_API_KEY: mapped to Authorization: Bearer ...

When using the customer-side Collector, configure tenant and API key in the Collector exporter instead. If the application still needs tenant metadata as a resource attribute, prefer OTEL_RESOURCE_ATTRIBUTES=elven.tenant.id=... so the app does not emit backend authentication headers.

Core OpenTelemetry variables:

• OTEL_SERVICE_NAME
• OTEL_SERVICE_VERSION
• OTEL_RESOURCE_ATTRIBUTES
• OTEL_EXPORTER_OTLP_ENDPOINT
• OTEL_EXPORTER_OTLP_PROTOCOL=grpc|http/protobuf
• OTEL_EXPORTER_OTLP_HEADERS
• OTEL_TRACES_SAMPLER
• OTEL_TRACES_SAMPLER_ARG
• OTEL_TRACES_EXPORTER=otlp

Serilog OpenTelemetry sink

ELVEN_LOG_BODY_MODE and ELVEN_REDACTION_MODE apply to logs that pass through the Elven/OpenTelemetry logging pipeline. A direct Serilog.Sinks.OpenTelemetry exporter sends records to the Collector before Elven processors can touch them, so raw SOAP/XML logged directly through that sink must be sanitized before logging or routed through the Elven logging pipeline.

If keeping a direct Serilog OTLP sink for a migration window, sanitize high-risk XML/JSON explicitly:

var safeXml = ElvenLogSanitizer.RedactBody(xml);
logger.Information("Sabre RQ {MensagemXML}", safeXml);

If you add a direct OTLP logging sink, keep ELVEN_SUPPRESS_OTLP_HTTP_SPANS=true or ELVEN_SUPPRESS_OTLP_EXPORTER_SPANS=true so the app does not create HttpClient or gRPC client spans for LogsService/Export.

• OTEL_METRICS_EXPORTER=otlp
• OTEL_LOGS_EXPORTER=otlp

OTEL_TRACES_EXPORTER=none, OTEL_METRICS_EXPORTER=none, OTEL_LOGS_EXPORTER=none, and OTEL_TRACES_SAMPLER=always_off are rejected in Elven mode. Logs, metrics, and traces are always on.

For http/protobuf, OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 is treated as a base endpoint. Elven expands it to /v1/traces, /v1/metrics, and /v1/logs per signal. If you provide a path explicitly, that path is preserved.

Log bodies are redacted by default with XML/JSON-aware structural redaction. Safe trace context fields such as trace_id, span_id, traceparent, and correlation_id are preserved.

Coverage

• Traces: ASP.NET Core, HttpClient, gRPC client, WCF client, CoreWCF server, SQL Client, EF Core, MongoDB source registration, RabbitMQ/Kafka/MassTransit source registration, AWS/Azure source registration, manual ActivitySource.
• Metrics: runtime, process, ASP.NET Core/Kestrel, HttpClient, WCF/CoreWCF RPC duration, DNS/TLS/connection meters, Elven internal metrics, trace-based exemplars.
• Logs: ILogger OTLP exporter, formatted message, scopes, state values, automatic trace/span correlation.
• Resources: service namespace/version/instance, deployment.environment.name, host, container, Kubernetes, process/runtime, tenant, region.
• Error mapping: cancellation, timeout, validation, auth, user input, HTTP 4xx/5xx, gRPC status codes, WCF fault/communication/security, DNS, connection refused, network, DB/provider, startup misconfiguration, unhandled exceptions.

Development

dotnet restore
dotnet build Elven.Observability.sln -c Release
dotnet test tests/Elven.Observability.UnitTests/Elven.Observability.UnitTests.csproj -c Release
dotnet pack Elven.Observability.sln -c Release -o artifacts/packages

Native AOT:

dotnet publish samples/NativeAot.WebApi/NativeAot.WebApi.csproj -c Release -r linux-x64

Strict trimming:

dotnet publish samples/NativeAot.WebApi/NativeAot.WebApi.csproj -c Release -r linux-x64 -p:PublishAot=false -p:PublishTrimmed=true -p:SelfContained=true

The facade package is optimized for convenience and broad instrumentation coverage. For strict Native AOT or trim-clean services, compose the lean core/exporter packages as shown in the Native AOT sample and avoid optional instrumentation modules that are not trim-clean in the upstream OpenTelemetry ecosystem.

Build the auto-instrumentation layer:

docker buildx build \
  -f docker/auto-instrumentation/Dockerfile \
  --platform linux/amd64,linux/arm64 \
  -t elvenobservability/dotnet-instrumentation:latest .

Build the OpenTelemetry Operator init-container image:

docker buildx build \
  -f Dockerfile.operator \
  --platform linux/amd64,linux/arm64 \
  -t elvenobservability/dotnet-instrumentation-operator:latest .

Operator setup lives in k8s/otel-operator-instrumentation.yaml. The image follows the OTel Operator .NET contract by exposing /autoinstrumentation with linux-x64 and linux-musl-x64 payloads plus the Elven plugin in /autoinstrumentation/plugins.

Run the local LGTM stack and send the samples to it:

docker compose -f docker-compose.lgtm.yml up -d
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_SERVICE_NAME=elven-local-sample
dotnet run --project samples/AspNetCore.WebApi/AspNetCore.WebApi.csproj

Open Grafana at http://localhost:3000; the bundled LGTM image receives traces, metrics, and logs over OTLP on 4317 and 4318.

References

• OpenTelemetry .NET SDK 1.15.3
• OpenTelemetry .NET AutoInstrumentation 1.15.0
• OpenTelemetry semantic conventions 1.41.0
• .NET 8, .NET 9, and .NET 10