We're looking to create a proper reporting solution on top of a tiered ABP template/application. We're not interested in searching/evaluating reporting components per se at this stage, but rather we're looking for a general approach to organizing reporting. Here are some inputs/given:
- The initial migration provisions four distinct databases for a solution (Identity management, Tenant mgmt, Application database, etc.)
- Every tenant in our app may have their own (separate) database instance (separate database, separate schema approach)
- The initial solution has a public site and the back-office part (for tenant/permission/feature management etc.)
We need to build some simple tenant-specific reports. Let's take the "file upload" functionality as a hypothetical example. Tenants need to know how many files they have uploaded daily/monthly/annually, etc. In this case, reporting is not a big deal as we can query this data from the tenant's database context. Reporting becomes a problem when we have hundreds of tenants with their own/dedicated database, and we (as a service provider) need to gather aggregated data from all those tenants (and display it on the back-office site). What's the recommended approach in this case? Creating a Data Warehouse? Also, querying against the live transactional DB is not a good idea, given that report generation may spoil the user experience and affect the app's performance. For this matter, a read-only replica for reporting solution would be a better idea, but it doesn't help much with many tenants who have a dedicated database. What's ABP's recommendation here?
5 Answer(s)
-
0
Hi,
Why not create a reporting service(module), it has a separate database or NoSQL database, like Elasticsearch? in this way you don't have to query hundreds of tenant databases to get reports.
About performance,
You can create a background worker to generate reports periodically, when you query, it's ready immediately. (not used for real-time reporting)
-
0
Hi,
Why not create a reporting service(module), it has a separate database or NoSQL database, like Elasticsearch? in this way you don't have to query hundreds of tenant databases to get reports.
About performance,
You can create a background worker to generate reports periodically, when you query, it's ready immediately. (not used for real-time reporting)
Hi! Could you elaborate on the first option, please? I didn't quite get how we benefit from the separate service + database (or ElasticSearch service). Do you have any useful links/references that may help us clarify this approach? Thanks!
-
0
Hi,
With independent services, the reporting module is more like an infrastructure, you can use it anywhere.
When you have hundreds of tenant independent databases, it's not good to query this data from the tenant's database context. It is better to use a separate database table for reporting.
Do you have any useful links/references that may help us clarify this approach
Sorry, I don't have such links. it's more like our audit logging module, you can check it.
-
0
Hi, How do we ensure that the data from many Tenant's databases go to Elasticsearch?
-
0
You can use the event bus (domain event or your custom event) to save data to DB or Elastic search.
If you already have some data, then you may need to do a data migration.
We have some usage of elasticsearch: https://github.com/abpframework/abp/blob/dev/modules/docs/src/Volo.Docs.Domain/Volo/Docs/Documents/FullSearch/Elastic/DocumentChangedEventHandler.cs https://github.com/abpframework/abp/blob/dev/modules/docs/src/Volo.Docs.Domain/Volo/Docs/Documents/FullSearch/Elastic/ElasticDocumentFullSearch.cs
