MongoDbRepository.Extensions.DataMesh.Bridge 6.4.3

MongoDbRepository.Extensions.DataMesh.Bridge

Enables real-time, low-latency Change-Data-Capture (CDC) replication data pipelines that sync operational MongoDB transactional tables into decoupled column-oriented analytical datastores, reporting views, or search indexes (e.g., ClickHouse, Snowflake, Elastic) without degrading production database performance.

🌟 Features

• Zero Transaction Impact: Relies entirely on out-of-band MongoDB Change Streams to tail changes asynchronously away from your main transaction engine.
• Resilient Connection Standby: Persists resume tokens inside an ObjectIdDocument database collection checkpoint layer to guarantee change-delivery tracking survives container recycles or pod updates.
• Auto-Provisioned Infrastructure: Integrates index providers straight into the core assembly index pipeline (AddAutomatedIndexes()) to establish unique data-mesh indexing fields automatically.

🚀 Quick Start

1. Model an Analytical Projection and Sink

Define the target reporting view model and implement the IDataMeshSink interface to push data to your analytical sink:

public record FinancialReportRow(string FacilityId, string CompanyName, decimal TotalExposure);

public class ClickHouseDataMeshSink : IDataMeshSink<FinancialReportRow>
{
    public async Task ProjectUpsertAsync(FinancialReportRow projection, CancellationToken cancellationToken)
    {
        // Execute real-time batch insert/update against ClickHouse/Data Lake cluster
    }

    public async Task ProjectDeleteAsync(string entityId, CancellationToken cancellationToken)
    {
        // Execute eviction tracking logic inside data lake table spaces
    }
}

2. Register the Bridge Engine Pipeline (Program.cs)

Map your rich corporate transactional entity into the lightweight analytical projection row using fluent LINQ selectors:

C#
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddMongoDbRepositoryCore(builder.Configuration.GetSection("MongoDb"));

// Bind a continuous CDC stream from 'CreditFacility' (transactional) into 'FinancialReportRow' (analytical)
builder.Services.AddDataMeshCdcBridge<CreditFacility, ObjectId, FinancialReportRow, ClickHouseDataMeshSink>(facility =>
    new FinancialReportRow(
        facility.Id.ToString(),
        facility.CorporateId,
        facility.ApprovedLimit
    ));

// Automatically builds structural checkpoint unique indexes across the system
builder.Services.AddAutomatedIndexes();