Use Stepping To Perform Atomic Multi-Step Operations

Stepping is a distributed BASE jobs implementation. You can use it as a workflow engine, event outbox/inbox, email/SMS sender, remote invoker, and more.

We have provided documentation for the following languages: English, 简体中文.

What are Job and Step in Stepping?

Job is a distributed transaction unit, and Step is a specific task inside a job.

A job contains one or many steps, and the transaction manager will execute them in order. If step 1 fails, it will be retried until success, and then step 2 starts to execute.

Scenarios for Using Stepping

Want To Execute Steps and Ensure Atomicity

When you start a job, Stepping will eventually complete the steps you require. If the app crashes during the executions, the transaction manager will continue to execute the rest steps after it recovers.

Stepping will complete your steps one by one. If a step fails, it will be tried later until success, which makes the job atomic. Please ensure all your steps can eventually succeed after retrying unless it is a Saga step.

Stepping may already complete the current step when your app crashes during the execution. When your app recovers, Stepping will execute it redundantly. Therefore, all your steps should be idempotent.

Want To Ensure Executing Steps After a DB Transaction Commits

When you start a job with a DB transaction, Stepping will eventually complete the steps you require after the DB transaction commits.

You don't need to worry about the non-atomicity caused by the app crashes after the transaction commits but before the steps' execution. We have handled this case by using the DTM's 2-phase messaging pattern.

Stepping also supports the "multi-tenant with multi-DB" scenario, meaning it works no matter how many different databases there are in your app.

Examples

The transaction manager will eventually complete the added steps:

var job = await distributedJobFactory.CreateJobAsync();

job.AddStep(new RequestBank1TransferOutStep(args)); // step with args
job.AddStep<RequestBank2TransferInStep>(); // step without args

await job.StartAsync();

The Steps document shows how to define a step.

If you want to execute the steps after a DB transaction commits and ensure they will eventually be done:

var db = serviceProvider.GetRequiredService<MyDbContext>(); // example for EF Core
await db.Database.BeginTransactionAsync();

var order = new Order(args);

db.Orders.Add(order);
await db.SaveChangesAsync();

var job = await distributedJobFactory.CreateJobAsync(new EfCoreSteppingDbContext(db));

job.AddStep(new SendOrderCreatedEmailStep(order));
job.AddStep(new SendOrderCreatedSmsStep(order));

await job.StartAsync(); // it will commit the DB transaction

Stepping supports EF Core, ADO.NET(coming soon), and MongoDB.

For details, please see the Usage document.

Installation

See the Installation document.

Supported Transaction Managers

Stepping requires transaction managers. You can choose an implementation you like.

DTM Server

DTM is a mature transaction manager you can use as the TM provider for Stepping. DTM allows you to use many other distributed transaction patterns like Saga, TCC, and XA.

See the DTM document.

Local-TM

Stepping provides a simple built-in TM implementation. The local-TM runs with your app as a local transaction manager. Which app starts a job should be the TM of this job.

See the Local-TM document.