DataArc.OrchestrationFramework 2.0.0

DataArc.OrchestrationFramework

Structured application orchestration, execution observation, and Entity Framework Core integration for .NET.

DataArc.OrchestrationFramework brings together DataArc application orchestration, event observation, and Entity Framework Core integration through a single composition package.

Applications can coordinate explicit application use-cases, execute persistence operations, observe application outcomes, and retain normal .NET dependency injection and standard Entity Framework Core usage.

Supports .NET 6 through .NET 10.

---

Free to Use

DataArc.OrchestrationFramework is available free of charge.

It may be used in personal, commercial, educational, government, and enterprise .NET applications without purchasing a DataArc commercial license.

dotnet add package DataArc.OrchestrationFramework

The framework composes the focused DataArc packages required for the complete application orchestration model.

Commercial SQL Server capabilities are available separately through DataArc.EntityFrameworkCore.SqlServer.

---

Framework Components

DataArc.OrchestrationFramework brings together the focused DataArc libraries.

DataArc.Orchestrator

Provides explicit, strongly typed application orchestration through ordinary .NET classes with visible dependencies and defined input and output contracts.

DataArc.Observer

Provides in-process event dispatch, execution observation, and independently registered event handlers.

DataArc.EntityFrameworkCore

Provides free composable parallel execution and high-performance bulk operations while preserving the normal Entity Framework Core programming model.

Supporting Infrastructure

Core DataArc packages provide the shared contracts and runtime infrastructure required by the composed components.

Each package remains independently defined and can also be consumed independently.

DataArc.OrchestrationFramework provides the composition layer.

---

Application Orchestration Flow

A DataArc application can coordinate application execution, persistence, and execution observation through an explicit application flow.

A typical workflow:

1. Receives strongly typed application input.
2. Executes an orchestrator.
3. Coordinates application services and persistence operations.
4. Returns a strongly typed result.
5. Publishes an application event based on the outcome.
6. Dispatches that event to independently registered observers.

Application Input
      |
      v
Orchestrator
      |
      v
Application / Persistence Coordination
      |
      v
Orchestrator Output
      |
      v
Application Event
      |
      v
DataArc.Observer

Each concern remains independently visible.

---

Define Input and Output

Application orchestration begins with explicit input and output contracts.

internal class OrchestratorInput : IOrchestratorInput
{
    public List<ForeignEntityExample> Data { get; set; } = new();
}

internal class OrchestratorOutput : IOrchestratorOutput
{
    public List<ForeignEntityExample> ForeignEntityExamples { get; set; } = new();

    public int AffectedRecords { get; set; }

    public bool Result { get; set; }
}

The input describes what the application operation requires.

The output describes what the orchestrator returns.

---

Define the Orchestrator

An orchestrator coordinates the application use-case through normal .NET dependencies.

internal class RepatriationOrchestrator
    : Orchestrator<OrchestratorInput, OrchestratorOutput>
{
    private readonly IDbContextFactory<DatabaseContextReadCommitted>
        _databaseContextReadCommittedFactory;

    public RepatriationOrchestrator(
        IDbContextFactory<DatabaseContextReadCommitted>
            databaseContextReadCommittedFactory)
    {
        _databaseContextReadCommittedFactory =
            databaseContextReadCommittedFactory;
    }

    public override async Task<OrchestratorOutput> ExecuteAsync(
        OrchestratorInput input,
        OrchestratorOutput output)
    {
        var primaryEntities = input.Data
            .Select(x => x.PrimaryEntityExample)
            .ToList();

        var foreignEntities = input.Data.ToList();

        await using var dbContext =
            await _databaseContextReadCommittedFactory.CreateDbContextAsync();

        var affected = await dbContext
            .AsParallel()
            .AddBulk(primaryEntities, 100_000, "prim")
            .AddBulk(foreignEntities, 100_000, "foreign")
            .SaveChangesParallelAsync();

        output.ForeignEntityExamples = foreignEntities;
        output.AffectedRecords = affected;
        output.Result = true;

        return output;
    }
}

The application orchestration remains explicit.

The orchestrator receives application input, coordinates the required work, and returns the application result.

Dependencies remain ordinary constructor-injected .NET services.

---

Register Application Orchestration

Register the orchestrators used by the application.

services.AddDataArcOrchestrator(orchestrators =>
{
    orchestrators.Add<RepatriationOrchestrator>();
});

There are no hidden handler chains between the caller and the orchestrator being executed.

---

Register Execution Observers

Observers can react independently to application outcomes.

services.AddDataArcObserver(observer =>
{
    observer.Add<DataRepatriationSuccessObserver>();
    observer.Add<DataRepatriationFailureObserver>();
});

Observation remains separate from the application use-case itself.

---

Execute the Application Use-Case

Execute the orchestrator using its explicit input and output contracts.

var repatriationResult = await _orchestrator
    .OrchestrateAsync<
        RepatriationOrchestrator,
        OrchestratorOutput>(
        new OrchestratorInput
        {
            Data = data
        },
        new OrchestratorOutput());

The caller explicitly selects the orchestrator and receives its defined output directly.

---

Publish the Application Outcome

The result of the application operation can be translated into an observable application event.

if (repatriationResult.Result)
{
    events.Add(
        new DataRepatriationSuccessEvent
        {
            Result = repatriationResult
        });
}
else
{
    events.Add(
        new DataRepatriationFailureEvent
        {
            Result = repatriationResult
        });
}

The primary application workflow does not need to know which observers will respond.

---

Dispatch Events

Dispatch application events through the observable event infrastructure.

await _observableEventHandler.DispatchAsync(events);

Registered observers receive applicable events independently.

---

Define Application Events

internal class DataRepatriationSuccessEvent : IObservableEvent
{
    public OrchestratorOutput Result { get; set; } = new();
}

internal class DataRepatriationFailureEvent : IObservableEvent
{
    public OrchestratorOutput Result { get; set; } = new();
}

---

Define Observers

Observers handle the application outcome without becoming part of the orchestrator itself.

internal class DataRepatriationSuccessObserver
    : IEventObserver<DataRepatriationSuccessEvent>
{
    public Task HandleAsync(
        DataRepatriationSuccessEvent evt)
    {
        Console.WriteLine(
            $"Success: {evt.Result.Result}");

        Console.WriteLine(
            $"Total Affected Records: {evt.Result.AffectedRecords:N0}");

        return Task.CompletedTask;
    }
}

internal class DataRepatriationFailureObserver
    : IEventObserver<DataRepatriationFailureEvent>
{
    public Task HandleAsync(
        DataRepatriationFailureEvent evt)
    {
        Console.WriteLine("Data repatriation failed.");

        return Task.CompletedTask;
    }
}

Execution observation remains outside the orchestrator while still participating in the overall application workflow.

---

Entity Framework Core Integration

DataArc.EntityFrameworkCore operates alongside standard Entity Framework Core usage.

Applications continue to use normal:

• DbContext
• IDbContextFactory<TContext>
• dependency injection
• LINQ
• Entity Framework Core queries
• Entity Framework Core transactions
• tracked persistence operations

DataArc capabilities can be introduced only where required.

---

Free Entity Framework Core Execution

DataArc.EntityFrameworkCore provides composable parallel execution and high-performance bulk operations free of charge.

Immediate bulk operations can execute directly against a standard DbContext:

await dbContext.AddBulkAsync(
    employees,
    100_000);

Standard entity operations and bulk operations can also be composed through the parallel execution builder:

var affected = await dbContext
    .AsParallel()
    .Add(entity)
    .AddBulk(data, batchSize)
    .SaveChangesParallelAsync();

The same DbContext remains available for ordinary Entity Framework Core operations before and after DataArc execution.

---

Commercial SQL Server Capabilities

Applications that require advanced SQL Server-specific functionality can add DataArc.EntityFrameworkCore.SqlServer.

Commercial capabilities include:

• Transaction-aware bulk inserts and deletes
• Participation in caller-owned Entity Framework Core transactions
• Transaction-coordinated parallel execution
• Composable transactional execution through AsParallelTransaction()
• Multi-context command coordination
• Multi-context query composition
• DataArc DDL Builder capabilities
• Relational model composition across multiple DbContext models
• SQL Server-specific advanced execution tooling

These capabilities require the applicable DataArc commercial entitlement and server initialization.

services.AddDataArcCore(options =>
    options.UseServerKey());

The DataArc.OrchestrationFramework package itself remains free.

---

Transactional Execution

With DataArc.EntityFrameworkCore.SqlServer, bulk operations can participate in an existing Entity Framework Core transaction.

await using var transaction =
    await dbContext.Database.BeginTransactionAsync();

await dbContext.AddBulkTransactionalAsync(
    employees,
    100_000);

await transaction.CommitAsync();

The application retains ownership of the transaction.

DataArc does not commit or roll back a caller-owned Entity Framework Core transaction.

For workloads requiring composable parallel execution with transactional guarantees:

var affected = await dbContext
    .AsParallelTransaction()
    .Add(entity)
    .AddBulk(data, batchSize)
    .CommitTransactionParallelAsync();

AsParallelTransaction() is provided by DataArc.EntityFrameworkCore.SqlServer.

It uses the same compositional builder model as AsParallel() while executing through the commercial SQL Server transactional path.

Transactional functionality is provided by the commercial SQL Server package rather than the free DataArc.EntityFrameworkCore package.

---

Database and DDL Composition

DataArc.EntityFrameworkCore.SqlServer includes the DataArc DDL Builder for applications that require deliberate control over how multiple Entity Framework Core models map to relational SQL Server databases.

Multiple DbContext models can participate in one physical relational database while remaining independently defined application boundaries.

Supported topologies include:

• Multiple DbContext models composed into one relational database
• Separate schemas within a shared database
• Cross-context foreign keys and relational constraints where appropriate
• Database-enforced relational integrity across composed models
• Multiple independent relational databases
• Separate connection strings for independently managed database models
• Full relational modelling within each independent database

A DbContext boundary does not have to become a relational boundary.

The application boundary and relational boundary remain separate architectural decisions.

---

Application Orchestration

DataArc application orchestration coordinates application operations around a defined use-case.

An orchestrator can coordinate:

• Application services
• Persistence operations
• Multiple DbContext instances
• Domain operations
• External services
• Data transformations
• Notifications
• Execution observation

The coordination remains visible in application code.

An application workflow is the specific sequence of operations coordinated for that use-case.

---

Package Architecture

DataArc.OrchestrationFramework is the composition layer for the free DataArc application orchestration stack.

DataArc.OrchestrationFramework
    |
    +-- DataArc.Orchestrator
    |
    +-- DataArc.Observer
    |
    +-- DataArc.EntityFrameworkCore
    |
    +-- supporting DataArc infrastructure

Implementation ownership remains with the focused packages.

DataArc.Orchestrator owns application orchestration.

DataArc.Observer owns event dispatch and execution observation.

DataArc.EntityFrameworkCore owns free Entity Framework Core parallel execution and bulk operations.

DataArc.EntityFrameworkCore.SqlServer is a separate commercial package that adds advanced SQL Server functionality where required.

DataArc.OrchestrationFramework composes the free application orchestration stack without requiring the commercial SQL Server package.

---

Licensing

DataArc.OrchestrationFramework is not a commercial DataArc product.

No commercial DataArc license, subscription, activation, or runtime entitlement is required to use the framework in accordance with its license.

Commercial licensing applies only when separately licensed DataArc functionality, such as capabilities provided by DataArc.EntityFrameworkCore.SqlServer, is used.

Installing or using DataArc.OrchestrationFramework does not grant rights to commercial DataArc functionality and does not require purchasing it.

See the included license file for full terms.

---

Documentation

Implementation guidance, architecture documentation, examples, benchmarks, and public demos are available through the DataArc documentation and repositories.

• Documentation: dataarc.dev
• Demo: DataArc.Orchestration.Framework.Demo public GitHub repository

---

Support

For support and commercial enquiries, contact support@dataarc.dev.

---

DataArc

Structured .NET tooling for complex application boundaries.