Introduce DTM for Multi-Tenant Multi-Database Scene

This article shares a new way to solve issue #10036 using the DTM's 2-phase messages pattern.

The module to be used today is EasyAbp's Abp.EventBus.Boxes.Dtm.

Introduction of the DTM Event Boxes Module

This implementation uses DTM's 2-phase messages to support ABP event boxes in the multi-tenant & multi-database scene.

You should see the DTM docs, which helps to understand this module.

Differences From the ABP's Default Event Boxes

DTM 2-phase Message Boxes ABP 5.0+ Default Boxes
Speediness :heavy_check_mark: :x:
Less data transfer :x: :heavy_check_mark:
Be guaranteed to publish
(transactional UOW)		 :heavy_check_mark: :heavy_check_mark:
Be guaranteed to publish
(non-transactional UOW)		 :x: :heavy_check_mark:
(consumers idempotency required)
Avoid duplicate handling
(with only DB operations)		 :heavy_check_mark: :heavy_check_mark:
Multi-tenant-database support :heavy_check_mark: :x:
No additional external infrastructure :x: :heavy_check_mark:
Dashboard and Alarm :heavy_check_mark: :x:

How Does the DTM Outbox Work?

You are publishing events using the ABP event outbox:

await _distributedEventBus.PublishAsync(eto1, useOutbox: true);
await _distributedEventBus.PublishAsync(eto2, useOutbox: true);  // The useOutbox is true by default.

The DTM outbox collects them temporarily. Let's see what it will do when you complete the current unit of work:

// Code snippet for UnitOfWork.cs
protected override async Task CommitTransactionsAsync()
{
    // Step 1: inserting a record to the DTM barrier table within the current DB transaction,
    //         and then it sends a "prepare" request to the DTM server.
    await DtmMessageManager.InsertBarriersAndPrepareAsync(EventBag);

    // Step 2: committing the current DB transaction.
    await base.CommitTransactionsAsync();

    // Step 3: sending a "submit" request to the DTM server.
    OnCompleted(async () => await DtmMessageManager.SubmitAsync(EventBag));
}

Now, the DTM server has received a "submit" request. It invokes the app's PublishEvents service with the events' data, and the latter will publish the events to the MQ provider immediately.

 

See the more detailed sequence diagram

 

 

If you are still confused about how it is guaranteed to publish, see DTM's 2-phase messages doc for more information.

How Does the DTM Inbox Work?

Unlike ABP's default implementation, the DTM inbox gets an event from MQ and handles it at once. After the handlers finish their work, the inbox inserts a barrier within the current DB transaction. Finally, it commits the transaction and returns ACK to MQ.

All the incoming events have a unique MessageId. Events with the same MessageId only are handled once since we cannot insert a barrier with a duplicate gid (MessageId).

 

See the more detailed sequence diagram

 

As you may have noticed, the inbox has nothing to do with the DTM Server.🤭

Installation and Usage

Please see https://github.com/EasyAbp/Abp.EventBus.Boxes.Dtm/tree/main#installation.

Postscript

This DTM event-boxes implementation is not a perfect solution. The main cost is that you need to care about the availability of the DTM Server. But if you run into the multi-tenant multi-database scene, it should be the best choice for now.