Interop.Db.Abstractions 1.0.1

Interop.Db.Abstractions - Seamless Data Synchronization for Entity Framework Core

Introduction

Interop.Db.Abstractions is a powerful NuGet package designed to augment Entity Framework Core applications with robust, built-in data synchronization capabilities. By providing an opinionated base DbContext (DbSyncContext) and a clear interface for sync-aware entities (IDbSyncEntity), this library streamlines the process of tracking changes, pushing updates, and pulling data from various external sources.

This library acts as the foundational layer for data synchronization. While it provides the core mechanisms for tracking and managing sync status within your database, it requires a concrete synchronization provider (such as Interop.Db.Telegram) to perform actual data transfer operations. Without a configured provider, Interop.Db.Abstractions will manage sync states internally but will not initiate external data synchronization.

Features

1. DbSyncContext: A specialized DbContext that extends standard EF Core functionality with automatic change tracking for synchronization purposes. It adds internal tables to manage synchronization states and intercepts SaveChanges calls to mark entities for sync.

2. IDbSyncEntity: A simple interface to mark your EF Core entities as sync-aware, ensuring they have a Guid Id for consistent tracking across sync operations.

3. AddOrUpdate Utility: A convenient method on DbSyncContext to effortlessly add new entities or update existing ones based on their IDbSyncEntity identifier.

4. Provider Agnostic: Designed to work with various synchronization providers, allowing you to choose the best fit for your application's external data sources (e.g., Telegram, custom APIs, etc.).

5. Minimal Schema Impact: Aims to avoid unnecessary changes to your existing database schema, only introducing internal tables and marking IDbSyncEntity entities for synchronization.

6. Flexible Usage: Allows normal DbContext operations while transparently extending functionality for sync management.

7. Extensibility: Provides interfaces (IDbSync) and helper classes for building custom synchronization providers tailored to unique requirements.

8. Change Notification: Offers a mechanism to listen for internal entity changes, enabling custom logic or further synchronization actions.

Installation

To use Interop.Db.Abstractions, install the NuGet package into your project

dotnet add package Interop.Db.Abstractions

You will also need an Entity Framework Core database provider (e.g., Microsoft.EntityFrameworkCore.Sqlite) and a specific synchronization provider (e.g., Interop.Db.Telegram).

Getting Started

1. Define Your Sync-Aware Entities

Your EF Core entities that you intend to synchronize must implement the IDbSyncEntity interface. This interface requires a Guid Id property, which DbSyncContext uses internally for tracking.

Example Entity (Message.cs):

using System;

namespace YourNamespace.Models
{
    /// <summary>
    /// Represents a message entity that can be synchronized.
    /// Implements IDbSyncEntity to enable sync tracking.
    /// </summary>
    public class Message : IDbSyncEntity
    {
        public Guid Id { get; set; }
        public string Value { get; set; }
        public string Owner { get; set; }
        public string To { get; set; }
        public DateTime Date { get; set; }
    }
}

2. Create Your Database Context

Instead of inheriting from DbContext, your application's database context must inherit from DbSyncContext. This provides the extended functionality required for synchronization management.

Example DbContext (TodoContext.cs):

using Interop.Db.Abstractions.Contexts; // Ensure this namespace is imported
using Microsoft.EntityFrameworkCore;
using YourNamespace.Models; // Your entity namespace

namespace YourNamespace.Data
{
    /// <summary>
    /// Your application's database context, inheriting from DbSyncContext.
    /// This enables internal synchronization tracking.
    /// </summary>
    public class TodoContext : DbSyncContext
    {
        public DbSet<Message> Messages { get; set; }

        public TodoContext(DbContextOptions<TodoContext> options) : base(options)
        {
        }

        // Optional: Configure your model as needed
        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder); // Always call base.OnModelCreating
            // Further model configuration if required
        }
    }
}

3. Register Services in Dependency Injection

Register your DbSyncContext and the Interop.Db.Abstractions services within your application's dependency injection container. This is typically done in Startup.cs (ASP.NET Core) or MauiProgram.cs (MAUI/Xamarin).

Example Registration (MauiProgram.cs):

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting; // For .UseMauiApp in MAUI
using Interop.Db.Abstractions.Extensions; // Required for AddDbSyncAbstractions()
using Microsoft.EntityFrameworkCore; // Required for UseSqlite or other EF Core providers
using System.IO; // Required for Path.Combine

// For MAUI applications:
public static class MauiProgram
{
    public static MauiApp CreateMauiApp()
    {
        var builder = MauiApp.CreateBuilder();
        builder.UseMauiApp<App>();

        // Configure your database path
        string dbPath = Path.Combine(FileSystem.AppDataDirectory, "mydatabase.db");

        // Register your DbContext, inheriting from DbSyncContext
        builder.Services.AddDbContext<TodoContext>(options =>
        {
            options.UseSqlite($"Filename={dbPath}"); // Or UseSqlServer Lite
        });

        // Register Interop.Db.Abstractions core services
        // IMPORTANT: This must be called to enable sync functionality
        builder.Services.AddDbSyncAbstractions();

        // Register your specific sync provider (e.g., Interop.Db.Telegram) here.
        // Without a provider, sync will be tracked internally but not sent/received externally.
        // builder.Services.AddTelegramDbSyncProvider(); // Example for Interop.Db.Telegram

        return builder.Build();
    }
}

4. Basic Usage

Once configured, you can interact with your DbSyncContext like a regular DbContext. The DbSyncContext will automatically intercept SaveChanges calls for IDbSyncEntity types and update the internal synchronization tables.

Example Usage:

using System;
using Microsoft.Extensions.DependencyInjection; // For CreateScope and GetRequiredService
using YourNamespace.Data; // Your DbContext namespace
using YourNamespace.Models; // Your entity namespace

public class Program
{
    public static async Task Main(string[] args)
    {
        var builder = MauiApp.CreateBuilder(); // Or Host.CreateDefaultBuilder() for console/ASP.NET
        builder.UseMauiApp<App>(); // If MAUI

        // ... (Service registration as shown above) ...

        var app = builder.Build();

        using (var scopeProvider = app.Services.CreateScope())
        {
            var context = scopeProvider.Services.GetRequiredService<TodoContext>();

            // Ensure the database is created and migrations are applied
            // In production, use migrations
            await context.Database.EnsureCreatedAsync();

            // Create a new message
            Message newMessage = new Message
            {
                Id = Guid.NewGuid(),
                Value = "Hello from Interop.Db.Abstractions!",
                Owner = "UserA",
                To = "UserB",
                Date = DateTime.Now
            };

            // Add the message to the DbSet
            context.Messages.Add(newMessage);

            // Save changes. This will automatically mark the entity for synchronization
            // in the internal sync tables due to DbSyncContext.
            await context.SaveChangesAsync();
        }
    }
}

Extensions and Utilities

DbSyncContext

Provides several utility methods crucial for managing synchronization, especially when building custom sync providers.

public partial class DbSyncContext : DbContext
{
    // ... (Standard DbContext properties and methods) ...

    /// <summary>
    /// Gets a ReflectionDbSet instance based on the DbSet name (e.g., "Messages").
    /// Useful for dynamic access to DbSets by string name.
    /// </summary>
    /// <param name="name">The name of the DbSet.</param>
    /// <returns>A ReflectionDbSet instance if found, otherwise null.</returns>
    public ReflectionDbSet? GetDbSetFromName(string name) { /* ... implementation ... */ }

    /// <summary>
    /// Gets a ReflectionDbSet instance corresponding to a given entity type.
    /// Useful for dynamically identifying the DbSet an entity belongs to.
    /// </summary>
    /// <param name="entityType">An instance of the entity type.</param>
    /// <returns>A ReflectionDbSet instance if found, otherwise null.</returns>
    public ReflectionDbSet? GetDbSetFromEntity(object entityType) { /* ... implementation ... */ }

    /// <summary>
    /// Saves all pending changes to the database. This method should ideally be called
    /// ONLY by synchronization providers to prevent infinite save loops and manage notifications.
    /// </summary>
    /// <param name="notify">If true, internal change notifications will be triggered.</param>
    public void SaveChangesFromSync(bool notify = true) { /* ... implementation ... */ }

    /// <summary>
    /// Adds a new entity or updates an existing one based on its IDbSyncEntity Id.
    /// This is a utility method to simplify upsert operations, especially for sync providers.
    /// </summary>
    /// <typeparam name="TDbSetSync">The type of the entity, which must implement IDbSyncEntity.</typeparam>
    /// <param name="entity">The entity to add or update.</param>
    /// <param name="updateOnly">An optional action to apply specific updates if the entity already exists.</param>
    public void AddOrUpdate<TDbSetSync>(TDbSetSync entity, Action<TDbSetSync>? updateOnly = null) where TDbSetSync : class, IDbSyncEntity { /* ... implementation ... */ }

    /// <summary>
    /// Logs a synchronization error internally within the DbSyncContext.
    /// Useful for providers to record issues during sync operations.
    /// </summary>
    /// <param name="message">The error message to log.</param>
    public void AddSyncError(string message) { /* ... implementation ... */ }
}

IDbSync Interface (For Sync Providers)

The IDbSync interface is the contract for synchronization providers. If you intend to create your own provider to integrate with external systems, you will implement this interface.

namespace Interop.Db.Abstractions
{
    /// <summary>
    /// Defines the contract for a data synchronization provider.
    /// Implement this interface to create concrete sync mechanisms (e.g., for Telegram, Web API).
    /// </summary>
    public interface IDbSync
    {
        /// <summary>
        /// Initiates a full synchronization cycle (Pull then Push).
        /// </summary>
        Task Sync();

        /// <summary>
        /// Starts the synchronization process, potentially listening for changes or scheduling tasks.
        /// </summary>
        Task Start();

        /// <summary>
        /// Stops the synchronization process.
        /// </summary>
        Task Stop();

        /// <summary>
        /// Pulls data from the external source into the local database.
        /// </summary>
        Task Pull();

        /// <summary>
        /// Pushes locally changed data to the external source.
        /// </summary>
        Task Push();
    }
}

When implementing IDbSync, you will have access to the DbSyncContext and its utility methods to interact with your local database entities and sync status.

Helper Classes for Synchronization Providers

Interop.Db.Abstractions provides several static helper classes that are invaluable when building synchronization logic, particularly within IDbSync implementations.

SyncChangesHelper

This helper class provides methods for querying and managing entities that have pending changes for synchronization.

namespace Interop.Db.Abstractions.Helpers
{
    public class SyncChangesHelper
    {
        /// <summary>
        /// Updates the internal SYNC table to mark any existing entities that
        /// are not yet tracked for synchronization (e.g., legacy data, or manual additions).
        /// This should be run periodically or on startup to ensure all IDbSyncEntities are tracked.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        public static void UpdateUntracked(DbSyncContext context) { /* ... implementation ... */ }

        /// <summary>
        /// Removes pending changes from the internal SYNC table associated with a specific SyncId.
        /// This is typically called after a successful push operation by a provider.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <param name="SyncId">The ID of the synchronization batch to remove.</param>
        public static void MarkAsSyncedRemoved(DbSyncContext context, int SyncId) { /* ... implementation ... */ }

        /// <summary>
        /// Marks a specific entity as successfully synchronized (no longer has pending changes).
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <param name="id">The Guid ID of the entity.</param>
        /// <param name="dbSetName">The name of the DbSet the entity belongs to.</param>
        /// <param name="SyncId">The synchronization batch ID associated with this sync operation.</param>
        public static void MarkAsSynced(DbSyncContext context, Guid id, string dbSetName, int SyncId) { /* ... implementation ... */ }

        /// <summary>
        /// Marks an entity as pending synchronization in the internal SYNC table.
        /// This is automatically handled by DbSyncContext.SaveChanges, but can be used
        /// for specific scenarios if needed.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <param name="entity">The entity to mark as pending sync (must implement IDbSyncEntity).</param>
        public static void MarkAsPendingSync(DbSyncContext context, object entity) { /* ... implementation ... */ }

        /// <summary>
        /// Retrieves a collection of all pending changes that need to be synchronized.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <returns>An IEnumerable of EntityChange objects representing pending additions, modifications, or deletions.</returns>
        public static IEnumerable<EntityChange> PendingChanges(DbSyncContext context) { /* ... implementation ... */ }
    }
}

SyncEntityHelper

This helper assists in common entity management operations for synchronization, such as deleting entities based on sync IDs or performing AddOrUpdate operations when data is pulled from an external source.

namespace Interop.Db.Abstractions.Helpers
{
    public class SyncEntityHelper
    {
        /// <summary>
        /// Deletes entities from their respective DbSets based on a given SyncId.
        /// This is typically used by providers to process deleted records from the remote source.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <param name="syncId">The SyncId identifying the batch of changes to delete.</param>
        public static void Delete(DbSyncContext context, int syncId) { /* ... implementation ... */ }

        /// <summary>
        /// Adds or updates an entity within the specified DbSet. This method is designed
        /// for use by sync providers when pushing data from the external system to the local database.
        /// It also updates the internal sync status accordingly.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <param name="entity">The entity to add or update (must implement IDbSyncEntity).</param>
        /// <param name="syncId">The SyncId associated with this incoming change.</param>
        public static void AddOrUpdate(DbSyncContext context, IDbSyncEntity? entity, int syncId) { /* ... implementation ... */ }
    }
}

SyncStatusHelper

This helper provides access to the overall synchronization state, allowing providers to track progress or timestamps.

namespace Interop.Db.Abstractions.Helpers
{
    public class SyncStatusHelper<T>
    {
        /// <summary>
        /// Retrieves the current overall synchronization state of the application.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <returns>The current SyncState, or null if not yet initialized.</returns>
        public static T? GetCurrent(DbSyncContext context) { /* ... implementation ... */ }

        /// <summary>
        /// Sets or updates the overall synchronization state.
        /// </summary>
        /// <param name="context">The DbSyncContext instance.</param>
        /// <param name="state">The custom state of the providers, it is serialized and saved in db.</param>
        public static void Set(DbSyncContext context, T state) { /* ... implementation ... */ }
    }
}

EntityChange and EntityChangeType

These models are used to describe the nature of a pending change for synchronization.

namespace Interop.Db.Abstractions.Models
{
    /// <summary>
    /// Represents a pending change to an entity that needs to be synchronized.
    /// </summary>
    public class EntityChange
    {
        /// <summary>
        /// The entity data
        /// Will be null for deleted entities.
        /// </summary>
        public object? Entity { get; set; }

        /// <summary>
        /// The entity data, represented as a dictionary of property names to values.
        /// Will be null for deleted entities.
        /// </summary>
        public Dictionary<string, object?> EntityDictionary { get; set; }

        /// <summary>
        /// The type of change (Added, Modified, Deleted).
        /// </summary>
        public EntityChangeType Type { get; set; }

        /// <summary>
        /// The name of the DbSet the entity belongs to.
        /// </summary>
        public string DbSetName { get; set; }

        /// <summary>
        /// The unique ID of the entity.
        /// </summary>
        public Guid Id { get; set; }

        /// <summary>
        /// The synchronization batch ID if the change originated from a specific sync operation.
        /// </summary>
        public int SyncId { get; set; }
    }

    /// <summary>
    /// Defines the type of change for an entity.
    /// </summary>
    public enum EntityChangeType
    {
        Added,
        Modified,
        Deleted
    }
}

Receiving Change Notifications

Synchronization providers or other parts of your application can subscribe to ISyncChanges.OnChangesSync to be notified when entities are detected as having changes that need to be synchronized. This typically happens after SaveChanges is called on DbSyncContext.

using Interop.Db.Abstractions; // For ISyncChanges
using Interop.Db.Abstractions.Models; // For EntityChange and EntityChangeType
using System.Collections.Generic;
using System.Linq;

// Somewhere in your application's setup (e.g., a service, or directly after app.Build()):
public class MySyncListener
{
    public MySyncListener(ISyncChanges changes)
    {
        // Subscribe to the change notification event
        changes.OnChangesSync += HandleChangesForSync;
    }

    private void HandleChangesForSync(IEnumerable<(object Entity, EntityChangeType Type)> changes)
    {
        // This method will be called when DbSyncContext detects changes.
        // You can use this to trigger your sync provider's Push method,
        // or perform other actions based on the changes.

        Console.WriteLine($"\nDetected {changes.Count()} changes for sync:");
        foreach (var change in changes)
        {
            Console.WriteLine($"- Entity Type: {change.Entity.GetType().Name}, Change Type: {change.Type}");

            // Example: If you need to map these changes to EntityChange objects for a provider
            // You might use SyncChangesHelper.PendingChanges() at this point,
            // or pass the raw changes to your provider.
        }

        // Example: If you have a sync provider instance available, you might trigger a push.
        // For demonstration, let's just log. In a real app, you'd likely inject and call
        // your IDbSync implementation.
        // _syncProvider.Push();
    }
}

// In MauiProgram.cs or Startup.cs, after app.Build():
// var app = builder.Build();
// var listener = new MySyncListener(app.Services.GetRequiredService<TodoContext>());

Internal Schema Changes

Interop.Db.Abstractions introduces the following internal tables to manage synchronization metadata:

1. SYNC: Tracks the Id and DbSet name of entities that have changes pending synchronization (HasChanges = true) or have been associated with a specific sync operation (SyncId).

2. SYNC_STATE: Stores the overall state of the synchronization process, including the last synchronization date and provider-specific timestamps (Pts, Qts).

3. SYNC_ERRORS: A log for errors encountered during synchronization operations.

These tables are managed internally by DbSyncContext and its helpers. You should not directly interact with them unless you are developing advanced custom synchronization logic.

Synchronization Providers (e.g., Interop.Db.Telegram)

Interop.Db.Abstractions is a framework; it does not perform actual data transfer by itself. You must integrate a specific synchronization provider to make use of its features. A provider like Interop.Db.Telegram would implement the IDbSync interface and handle the logic for pushing local changes to, and pulling remote changes from, a Telegram-based system.

Without a registered IDbSync provider, Interop.Db.Abstractions will simply track changes internally but will not initiate any external data synchronization.