Open Closed

How to merge multiple tenant databases into a single database #7802


User avatar
0
pablo@ccalp.net created
  • ABP Framework version: v7.X.X
  • UI Type: Angular
  • Database System: EF Core (SQL Server)

Hello, I'm trying to come up with a good solution to merge multiple tenant databases into a single one for reporting, my first approach is to use the entity synchronizer provided by the framework, I think it would be a good solution, but I'm not sure how to solve some issues and I need some guidance. Here are the requirements:

  • Synchronize data from multiple tenant databases into a single one
  • Support for multiple tenant database groups, Tenants A, b, and C go to Group 1, Tenants D, E, and F go to Group 2.
  • Be able to create the report databases and migrate them using the framework features (Host, DbMigrator)
  • When synchronizing the entities, be able to specify in which report database (Group 1, Group 2, etc.) should they be created, updated, or deleted from

Any help would be greatly appreciated.


12 Answer(s)
  • User Avatar
    0
    pablo@ccalp.net created

    I forgot to ask about how can I filter events by tenantId, for example, if I want to use the entity synchronizer to publish entity change for some tenants and not all of them.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    I'm trying to come up with a good solution to merge multiple tenant databases into a single one for reporting

    You can switch the current tenant ID to query the tenant's data. Have you tried this?

  • User Avatar
    0
    pablo@ccalp.net created

    Yes, I can do that when the event is received, but I have not found how to do a cancellation or prevent the event from being published for some tenants so I can prevent those messages from going out.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    I mean you do not need to merge multiple tenant databases into a single database, You can just switch to query the data of all tenant and host.

  • User Avatar
    0
    pablo@ccalp.net created

    I need to create reports in real-time aggregating multiple tenants that belong to the same company, switching tenants to gather the data in memory and then aggregating it would not be efficient.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    Then you should use the database function or tool to synchronize multiple databases instead of using code .

    If you want to use code to do this, You can check this: https://chrobyte.de/blogs/default/abp-module-development-and-cross-module-dependencies

    https://abp.io/docs/latest/framework/infrastructure/event-bus/distributed?_redirected=B8ABF606AA1BDF5C629883DF1061649A#entity-synchronizer

  • User Avatar
    0
    pablo@ccalp.net created

    I started using CDC (change data capture) in Azure Data Factory, but the cost is too high, so I need to implement it in code. Thanks.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    ok

    You can check this: https://chrobyte.de/blogs/default/abp-module-development-and-cross-module-dependencies

    https://abp.io/docs/latest/framework/infrastructure/event-bus/distributed?_redirected=B8ABF606AA1BDF5C629883DF1061649A#entity-synchronizer

  • User Avatar
    0
    pablo@ccalp.net created

    Any help with how to prevent sending a message based on the Tenant ID? I have configured the distributed entity event options and added an auto event selector for my entities in the publisher application and I am receiving and filtering out the events fine in the subscriber based on the tenant Id, but I would really like to not send messages that should not be processed in the subscriber.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    Currently not supported. You have to check the tenant ID on the handler.

  • User Avatar
    0
    pablo@ccalp.net created

    Got it, thank you.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    : )

Made with ❤️ on ABP v9.0.0-preview Updated on September 19, 2024, 10:13