SchemaForge.Abstractions 2.1.1

SchemaForge

A high-performance Cross-Database Migration Engine for .NET that ports schemas and data between different database vendors. Designed for enterprise-scale modernizations, SchemaForge handles complex views, cross-schema foreign keys, and massive datasets with ease.

💡 Engine vs. Tool: When to use SchemaForge?

It is important to distinguish SchemaForge from "Schema Evolution" tools like Liquibase or Flyway:

• Liquibase / Flyway are for Version Control. Use them to manage incremental changes (v1 → v2) within the same database vendor over time.
• SchemaForge is a Porting Engine. Use it to move an entire system from one vendor to another (e.g., SQL Server → PostgreSQL). It automates the complex translation of data types, SQL dialects for views, and handling of legacy data orphans.

Supported Databases

• SQL Server (source and target)
• PostgreSQL (source and target)
• MySQL (source and target)
• Oracle (source and target)

Features

• Robust Schema Migration - Automatic creation of tables, primary keys, and complex foreign key relationships across all 4 platforms.
• Enterprise-Grade Constraints - Implements "Future Enforcement, Past Forgiveness" model using ENABLE NOVALIDATE (Oracle), NOT VALID (Postgres), and WITH NOCHECK (SQL Server) to ensure migrations complete even with legacy data orphans.
• Automated Infrastructure - Automatically creates target databases, schemas, and Oracle users/permissions on-the-fly.
• High-Performance Data Transfer - Parallel table migration with batched bulk writes, read-ahead pipelines, and sql_log_bin=0 (MySQL) for maximum speed and disk efficiency.
• Intelligent View Migration - Sophisticated SQL dialect conversion for view definitions, handling complex functions like DECODE, BITAND, SYSDATETIME, and varied quoting styles.
• Verified with WideWorldImporters - Stress-tested and verified using the full, multi-schema production dataset from Microsoft.
• Plugin-Based Architecture - Fully extensible provider model built on .NET 9 and standard interfaces.
• Dry Run Mode - Generate and preview complete migration SQL scripts without executing against the target.

Quick Start

Install as a .NET global tool

# Install from NuGet
dotnet tool install --global SchemaForge.Cli

# Run a migration
schemaforge --from postgres --to oracle \
  --source-conn "Host=127.0.0.1;Database=prod;Username=pg;Password=..." \
  --target-conn "User Id=system;Password=...;Data Source=127.0.0.1:1521/FREEPDB1" \
  --schema SALES --verbose

Use as a library in your .NET app

dotnet add package SchemaForge
dotnet add package SchemaForge.Providers.SqlServer
dotnet add package SchemaForge.Providers.Postgres

using SchemaForge.Builder;

await DbMigrate
    .FromSqlServer(srcConn)
    .ToPostgres(targetConn, "public")
    .MigrateAll()
    .WithMaxParallelTables(4)
    .ExecuteAsync();

CLI Usage

Data Type Mapping

Changelog

v2.1.1 — Resource Leak & FK Naming Fixes

• MigrationBuilder ServiceProvider Disposal — ExecuteAsync now disposes the ServiceProvider (and its connection pool) in a finally block. Previously, calling ExecuteAsync in a loop (e.g., once per schema) accumulated undisposed pools, causing connection exhaustion on large multi-schema migrations.
• Batch DataTable Disposal — BulkDataMigrator disposes each batch DataTable immediately after the write step instead of waiting for GC. Eliminates memory growth proportional to total row count on large datasets.
• FK Naming Double-Conversion Fixed — SqlServerSchemaWriter.GenerateAddForeignKeySql was applying namingConverter.Convert() to the FK name after prepending FK_, resulting in FK_FK_ prefixes on round-trip migrations. The second convert call is removed; the name is final after prefix prepending.
• SqlServerDataReader COUNT(*) Timeout — GetRowCountAsync now sets CommandTimeout = 0 (unlimited) to handle large tables whose cold-scan COUNT(*) exceeds the 30-second default.
• docker-compose.test.yml — Switched SQL Server image from azure-sql-edge to mssql/server:2022-latest; capped memory at 2 GB; updated healthcheck to mssql-tools18 path.
• SchemaForge.IntegrationTests — New integration test project added to the solution.

v2.1.0 — 12-Way Matrix Validation & Index Fidelity

• DatabaseType Enum — Replaced string-based DB type comparisons throughout the internal pipeline with a strongly-typed DatabaseType enum and ToEnum() / ToProviderKey() extension methods.
• --preserve-names Flag — New CLI flag (and .PreserveNames() fluent method) to retain original identifiers instead of applying target DB naming conventions (snake_case / PascalCase / UPPERCASE).
• Oracle CHECK Constraint Fix — Oracle stores CHECK expressions with "COLUMN" double-quoted identifiers. When migrating to Postgres these caused a zero-length delimited identifier error. Fix gates on source DB detection and strips Oracle double-quotes before identifier conversion.
• SYSTIMESTAMP Detection — Added SYSTIMESTAMP to Oracle indicators in DetectSourceDatabase so Oracle DEFAULT expressions are correctly identified and converted (previously fell through to SQL Server handling).
• Postgres Index Fidelity — PostgresSchemaReader now excludes indexes that back a pg_constraint record, preventing unique-constraint backing indexes from being double-counted as standalone indexes during round-trip migrations.
• MySQL Index Fidelity — MySqlSchemaReader now excludes FK auto-backing indexes (MySQL silently creates an index for every foreign key column). Previously inflated index counts by ~39 when migrating from MySQL.
• Full 12-Path Matrix Validated — All 12 source→target migration combinations verified against the Microsoft WideWorldImporters dataset (993,095 rows across 45 tables, 5 schemas) with row counts confirmed in sync across SQL Server, PostgreSQL, MySQL, and Oracle.

v2.0.3 — CLI Stability Patch

• Fixed Dry Run Output — Resolved an issue where the --dry-run-output flag failed to save the generated SQL script to disk.
• Improved Boolean Consistency — Standardized tinyint to boolean mapping for PostgreSQL to ensure view compatibility.
• Enhanced MySQL Discovery — Fixed schema/database resolution in MySqlSchemaReader.

v2.0.2 — Enterprise Resilience Update

• Major Release — Full 12-path verification matrix complete with WWI dataset.
• Robust FKs — Implemented ENABLE NOVALIDATE (Oracle), NOT VALID (Postgres), and WITH NOCHECK (SQL Server).
• MySQL Driver Upgrade — Switched to MySqlConnector for superior Docker/SSL stability.
• Infrastructure Automation — Added EnsureDatabaseExistsAsync for SQL Server and MySQL.
• Dialect Expansion — Added support for BITAND, DECODE, and FROM_TZ translations.
• API Enhancement — New WithMaxParallelTables(n) and WithSourceSchemaFilter fluent methods.
• Performance — Implemented sql_log_bin=0 for MySQL to prevent "Disk Full" errors during large moves.
• Bug Fixes — Fixed 15+ critical bugs in view routing, regex backreferences, and identifier quoting.

v1.0.51

• Fixed SQL Server to SQL Server migration issues and cross-schema FK references.