JobMaster 0.0.5-alpha

JobMaster .Net

Distributed job orchestration engine for .NET. Built for horizontal scale.

JobMaster is a high-performance framework designed to manage and execute background tasks across a distributed cluster. By decoupling coordination from execution, it allows developers to scale compute resources horizontally based on real-time workload demands without compromising system stability.

📦 Installation & NuGet

dotnet add package JobMaster

🏗 Core Architecture

To achieve true resilience and massive scale, JobMaster utilizes a three-layer architecture:

1. The Master (Coordination): The "Source of Truth." It manages cluster topology, job definitions, and long-term auditing. It ensures that the state of your entire ecosystem is persistent and consistent.
2. Agents (Transport): High-speed ephemeral storage or message brokers (PostgreSQL, SQL Server, or NATS JetStream). Agents act as high-performance buffers for "in-flight" jobs, ensuring workers can claim tasks with ultra-low latency.
3. Workers (Execution): The compute power. Workers monitor specific Agents, claim available jobs using atomic locks, and synchronize execution results back to the Master Database.

🚀 Getting Started

1. Configuration

Register JobMaster in your Program.cs. This defines your cluster identity and sets up your storage providers.

builder.Services.AddJobMasterCluster(config =>
{
    // Define the central coordination database
    config.ClusterId("Production-Cluster")
          .UsePostgresForMaster(connectionString);

    // Define the transport layer (Agent)
    config.AddAgentConnectionConfig("Transport-1")
          .UsePostgresForAgent(agentConnectionString);
    
    // Attach a worker to the transport
    config.AddWorker()
          .AgentConnName("Transport-1");
});

// Start the runtime
await app.Services.StartJobMasterRuntimeAsync();

Implement a Job Handler

Define your logic by implementing IJobHandler

public class NotificationHandler : IJobHandler
{
    public async Task HandleAsync(JobContext job)
    {
        var userId = job.MsgData.GetStringValue("UserId");
        // Business logic here...
        await Task.CompletedTask;
    }
}

Schedule Jobs

Inject IJobMasterScheduler into your services or Minimal APIs.

Immediate or Delayed Execution

app.MapPost("/send-welcome", async (IJobMasterScheduler scheduler) =>
{
    var msg = WriteableMessageData.New().SetStringValue("UserId", "user_123");
    
    // Enqueue for immediate processing
    await scheduler.OnceNowAsync<NotificationHandler>(msg);
    
    return Results.Accepted();
});

Recurring Schedules (Static & Dynamic)

JobMaster supports both code-defined (Static) and runtime-defined (Dynamic) recurring tasks.

Simple Interval

var msg = WriteableMessageData.New().SetStringValue("UserId", "user_123");

await scheduler.RecurringAsync<NotificationHandler>(
    TimeSpan.FromHours(24), 
    msg);

NaturalCron

using NaturalCron;
using JobMaster.RecurrenceExpressions.NaturalCron;

var msg = WriteableMessageData.New().SetStringValue("UserId", "user_123");
// Via Expression
await scheduler.RecurringAsync<NotificationHandler>(
    NaturalCronExprCompiler.TypeId,
    "every day between mon and fri at 18:00",
    msg);

// Via fluent builder
NaturalCronExpr schedule = NaturalCronBuilder
.Every(30).Minutes()
.In(NaturalCronMonth.Jan)
.Between("09:00", "18:00")
.Build();

await scheduler.RecurringAsync<NotificationHandler>(
    schedule,
    msg);

See NaturalCron

🛠 Advanced Features

• Transport Agnostic: Seamlessly switch between RDBMS (Postgres, MySQL, SQL Server) and Message Brokers (NATS JetStream) without changing your business logic.
• Performance Buffering: New jobs are instantly persisted to the Agent for immediate execution, then asynchronously synced to the Master for long-term auditing.
• Atomic Locking: Built-in protection to ensure that even in a multi-node cluster, a job is never executed by more than one worker simultaneously.
• Static & Dynamic Scheduling: Support for code-defined "Static Profiles" that sync on startup and "Dynamic Schedules" created at runtime via API.
• Horizontal Scaling: You can spin up as many worker instances as needed to handle your current workload without reconfiguring the Cluster Database.