Microservice Startup Template: Database Migrations
Running SQL-Server infrastructural service is required since default connection strings for microservices use SQL-Server running on container.
There are two ways to migrate databases and seed data for microservices. Either one of them or both of them can be used as long as data seeders are synced.
Auto Migration (On-the-fly Migration)
RabbitMq infrastructural service is required for auto migration along with SQL-Server.
The aspect of auto migration is, that each microservice migrates its own database and seeds its own data. To achieve this, each microservice checks pending migration and handles it on its own. To achieve that, microservices acquire a distributed lock and apply the database migration to prevent multiple replicas to try migrating the already migrated database.
Microservices have a DbMigrations folder that contains DatabaseMigrationChecker and DatabaseMigrationEventHandler specific to that microservice. Since this behavior is shared across all the microservices, it has been defined as PendingEfCoreMigrationsChecker
.
Each microservice extends the PendingEfCoreMigrationsChecker
and runs the DatabaseMigrationChecker at OnPostApplicationInitialization
method.
Each migration tries to lock and apply database migrations under PendingEfCoreMigrationsChecker
base class:
The retry mechanism is implemented under the TryAsync
method if the database is not reachable or not ready yet. By default, it has 3 retry counts and this can be modified under the PendingMigrationsCheckerBase
. The distributed lock is used to prevent multiple concurrent applications (replicas) are trying to migrate the same database.
Under the LockAndApplyDatabaseMigrationsAsync
method, when the distributed lock is acquired; the database is migrated based on pending migrations:
On-the-fly migration is achieved by using the distributed event system by handling the ApplyDatabaseMigrationsEto
distributed event in the DatabaseMigrationEventHandler
.
After the host database migrations by queueing a new ApplyDatabaseMigrationsEto
event passing TenantId argument.
This is handled under MigrateDatabaseSchemaAsync(Guid? tenantId)
method of DatabaseMigrationEventHandlerBase.
With this feature; the newly created tenant with a specified connection string will have its new database automatically created with all the tables and the initial seed data if available. Tenant management UI also has a manual trigger for database creation & migration in case any problem occurs with automatic migration.
If you don't want to use Auto Migration for any microservice, delete the
OnPostApplicationInitialization
method of the HttpApiHost module and microservice-specific DatabaseMigrationChecker file along with related DataSeeder.
Database Migrator
If you want to use only Auto Migration (On-the-fly-Migration), it is safe to delete this project completely.
DbMigrator is a console application and used for database migrations and seeding data. It is located under the shared folder in the microservice template solution. DbMigratorModule
depends on SharedHostingModule
to use mapped database configurations. Since it will be migrating the databases of microservices; it also depends on each microservice's EntityFrameworkCoreModule
and ApplicationContractsModule
modules.
DbMigratorService first migrates the host then the tenants.
MigrateHostAsync
method migrates all the databases.
MigrateTenantsAsync
method migrates each tenant based on their connection string
If you decide to use both DbMigrator and Auto Migration approaches, you need to keep duplicate dataseeder files; one under your microservice DbMigrations folder for auto migration and the other under the shared DbMigrator project. You need to keep both of your data seeder files synchronized.
IdentityService Data Seeding
IdentityService uses three different mapped database configurations; IdentityService, AdministrationService and SaasService which are located under appsettings.json file.
IdentityService seeding is required for AuthServer since it seeds the admin user/password (identity data) and initial OpenIddict data (applications, scopes).
OpenIddict Data Seeding
Both Auto Migration Data Seeding and DbMigrator Data Seeding use OpenIddictDataSeeder
. The seeder uses OpenIddict section in appsettings.json file to seed client and resource cors and redirectUri data.
Creating API Scopes
Each scope is defined individually.
Creating Web Gateway Swagger Client
Web Gateway Swagger client is the only swagger client used for all swagger authorization in between microservices and gateways.
Creating Applications
While public-web and administration service clients are distinct, all the other back-office clients are created by default. Administration service is used to make requests to identity service to get user permission data. See administration service for more.
If you have created an angular back-office application in your microservice template, you are free to delete blazor server, blazor and web clients.
Auto Migration Data Seeding
IdentityServiceDatabaseMigrationEventHandler
is used for seeding language management and identity data.
Data seeding occurs when the database is migrated:
Or when a tenant is created:
Or when a tenants connection string is updated:
Changing the connection string will cause the creation of a new database. This database will be empty if data from the old database is not moved to the new database. It is left for the developer's choice.
DbMigrator Data Seeding
Host data is seeded in MigrateHostAsync
method:
IdentityServiceDataSeeder
is used for seeding the required OpenIddictDataSeeder
and seeding data runs after migrating all the databases:
Tenant data is seeded after host data is migrated in the MigrateTenantsAsync
method after tenant database migration is completed:
If you decide to use both DbMigrator and Auto Migration approaches, you need to keep both
IdentityServerDataSeeder
files located under IdentityService/DbMigrations and DbMigrator projects synchronized.
AdministrationService Data Seeding
AdministrationService uses two different mapped database configurations; AdministrationService and SaasService which are located under appsettings.json file. AdministrationService seeds the initial admin permission data.
Auto Migration Data Seeding
AdministrationServiceDatabaseMigrationEventHandler
is used for seeding permissions:
Data seeding occurs when database is migrated:
Or when a tenant is created:
Or when a tenants connection string is updated:
DbMigrator Data Seeding
IDataSeeder is used for seeding the required data.
SaasService Data Seeding
SaasService uses two different mapped database configurations; SaasService and AdministrationService which are located under appsettings.json file. SaasService does not seed any initial data. If an initial Saas data is required to be seeded, create a SaasDataSeedContibutor.
SaasService database is only available for the host side.
Auto Migration Data Seeding
SaasServiceDatabaseMigrationEventHandler
is used for seeding initial Saas data by calling SeedAsync
method of the newly created SaasDataSeedContibutor after migrating the database schema.
SaasServiceDatabaseMigrationEventHandler only handles
ApplyDatabaseMigrationsEto
by default.
DbMigrator Data Seeding
IDataSeeder is used for seeding the required data and Data seed contributors are automatically discovered by the ABP Framework and executed as a part of the data seed process.
ProductService Data Seeding
ProductService uses three different mapped database configurations; ProductService, AdministrationService, and SaasService which are located under appsettings.json file. ProductService does not seed any initial data. If an initial product data is required to be seeded, create a ProductDataSeedContibutor.
Auto Migration Data Seeding
ProductServiceDatabaseMigrationEventHandler
is used for seeding initial product data by calling SeedAsync
method of the newly created ProductDataSeedContibutor after migrating the database schema.
If the product data needs to be seeded when a tenant is created:
Or when a tenants connection string is updated:
DbMigrator Data Seeding
Generic IDataSeeder is used to seed the required data and Data seed contributors are automatically discovered by the ABP Framework and executed as a part of the data seed process.