SQLBulkProcessor 1.0.8

SQLBulkProcessor

High-performance bulk Insert, Update, Upsert, Delete, and Merge for Entity Framework Core on SQL Server.

The library maps entities from your EF Core model, streams them with SqlBulkCopy, and applies set-based SQL (UPDATE/DELETE/MERGE) against a session-scoped temp table. It is designed as a drop-in NuGet package for EF Core 8 projects.

Requirements

• .NET 8
• Entity Framework Core 8 with the SQL Server provider
• SQL Server (Microsoft.Data.SqlClient)

Install

Project reference (local pack):

dotnet pack src/SQLBulkProcessor/SQLBulkProcessor.csproj -c Release
dotnet add package SQLBulkProcessor --source ./src/SQLBulkProcessor/bin/Release

Or add a project reference:

<ItemGroup>
  <PackageReference Include="SQLBulkProcessor" Version="1.0.0" />
</ItemGroup>

Quick start

using Microsoft.EntityFrameworkCore;
using SQLBulkProcessor;

await db.BulkInsertAsync(products);
await db.BulkUpdateAsync(products);
await db.BulkUpsertAsync(products);
await db.BulkDeleteAsync(products);
await db.BulkMergeAsync(products);

Sync overloads (BulkInsert, BulkUpdate, BulkUpsert, BulkDelete, BulkMerge) are also available.

Empty collections are a no-op and return 0, except BulkMerge with delete enabled (the default), which throws so it cannot wipe the target table.

Operations

BulkMerge is a table sync. To merge without deleting unmatched target rows, use BulkUpsert or set DeleteWhenNotMatchedBySource = false.

Options

await db.BulkInsertAsync(products, options =>
{
    options.BatchSize = 5_000;
    options.TimeoutSeconds = 120;
    options.UseTableLock = true;
    options.OutputIdentity = true;   // write generated IDs back onto entities
    options.KeepIdentity = false;    // set true to insert explicit ID values
    options.ExcludeColumns = ["CreatedAt"];
    options.OnRowsCopied = copied => Console.WriteLine($"Copied {copied}");
});

await db.BulkUpsertAsync(products, options =>
{
    options.KeyColumns = ["Sku"];    // match on a business key instead of PK
    options.InsertWhenNotMatched = true;
    options.UpdateWhenMatched = true;
});

await db.BulkMergeAsync(products, options =>
{
    options.DeleteWhenNotMatchedBySource = true;
});

Column filters (IncludeColumns, ExcludeColumns, KeyColumns) accept property names or SQL column names.

Transactions

Operations enlist in the current EF Core transaction and connection:

await using var tx = await db.Database.BeginTransactionAsync();
await db.BulkInsertAsync(newProducts);
await db.BulkUpdateAsync(changedProducts);
await tx.CommitAsync();

Mapping notes

• Table, schema, keys, identity, computed, rowversion, and temporal period columns come from the EF Core model.
• TPH discriminators are included. Mixed TPH collections of a base type are supported.
• Owned reference types mapped to the same table are included. Owned collections are not.
• Value converters and enums are applied when reading entity values.
• Change tracker is not used. Entities are not attached or marked Unchanged after a bulk call.
• Source rows for upsert/merge must be unique on the match key (MERGE fails on duplicate source keys).

Benchmark

The test project includes a SQL Server catalog benchmark with a deterministic seed. Each operation is timed twice on the same in-memory dataset:

• EF ChangeTracker — standard AddRange / UpdateRange / RemoveRange / SaveChanges, plus load-and-mutate for upsert and merge
• SQLBulkProcessor — BulkInsert / BulkUpdate / BulkUpsert / BulkDelete / BulkMerge

Setup (truncate / reload) is excluded from the timings so the comparison is the operation itself. Measured on SQL Server (local instance), .NET 8, Release. ChangeTracker is 1 iteration; bulk is the mean of 3 iterations after a 2,000-row warmup.

25,000 rows

On this dataset, bulk insert, update, upsert, and merge finished in well under a second while ChangeTracker took 6–21 seconds. Delete is closer (ChangeTracker already issues relatively cheap DELETE statements by key) but bulk is still more than twice as fast.

200,000 rows

Same generator, seed, and column mix, scaled to 200,000 rows.

Speedups hold at 8× the row count. Insert, update, upsert, and merge stay in the 17–30× range; ChangeTracker merge takes about 2.7 minutes while bulk finishes in about 6 seconds.

dotnet test tests/SQLBulkProcessor.Tests -c Release --filter Category=Benchmark
dotnet test tests/SQLBulkProcessor.Tests -c Release --filter Size=200k

The tests are skipped when SQL Server is not reachable. Override the connection with SQLBULKPROCESSOR_CONNECTION. Exclude them from a normal run with --filter Category!=Benchmark.

SQL Server integration tests (insert / update / upsert / delete / merge against a real database) run with the default test suite when SQL Server is available, or explicitly:

dotnet test tests/SQLBulkProcessor.Tests -c Release --filter Category=Integration

Pack

dotnet pack src/SQLBulkProcessor/SQLBulkProcessor.csproj -c Release

The package includes this README, an MIT license, and a symbol package (.snupkg).

Merges into main pack and publish through GitHub Actions (.github/workflows/build.yml). nuget.org uses Trusted Publishing (OIDC) rather than a long-lived API key. The nuget.org policy must list this repository, the workflow file build.yml, and the GitHub Environment production. Set the Actions variable NUGET_USER to your nuget.org profile name if it is not the same as the GitHub owner.