Migrations EFcore loop issue #1807 | Support

dmeagor created 3 years ago

ABP Framework version: Any
UI type: Any
DB provider: EF Core
Tiered (MVC) or Identity Server Separated (Angular): Any

I have previously reported this as a bug here. https://github.com/abpframework/abp/issues/9467

ABP EFCore migrations are currently running N+1 loops with the tenant seeding without recreating the dbContext instance for each loop. This is against microsoft's EFCore guidence and causes severe slowdowns after a few hundred tenants. I believe this is due to change tracking.

This seems fairly simple to fix but it's not something we can do here. Possible fixes are.

Don't do long iterations. Pull out permissions etc. in one query. Looking at PermissionDataSeeder.cs this seems like a quick change.
Recreate the dbContext object inside of the loop for each iteration ( each tenant )
Turn off change tracking for the dbContext instance or specific queries.

The quick fixes seem to be disabling change tracking or recreating the dbContext on each tenant loop. I also think pulling out the permissions in bulk rather than one at a time would make a big difference.

For our number of users is it entirelly impossible to migrate the database seeding (structural changes are fast though.)

I submitted this to github and to bugs and issues and got no response, just closed as stale.

4 Answer(s)

0

liangshiwei created 3 years ago
Support Team Fullstack Developer

Hi,

I will check it out.
0

liangshiwei created 3 years ago
Support Team Fullstack Developer

Don't do long iterations. Pull out permissions etc. in one query. Looking at PermissionDataSeeder.cs this seems like a quick change.

I will try impove it, see https://github.com/abpframework/abp/pull/9978

Recreate the dbContext object inside of the loop for each iteration ( each tenant )

Database context instance is expensive, we should reuse

Turn off change tracking for the dbContext instance or specific queries.

Repository is abstract, you can do it in your project. see : https://github.com/abpframework/abp/issues/9652
0

dmeagor created 3 years ago

Thanks for your help with this!

Recreate the dbContext object inside of the loop for each iteration ( each tenant )

Database context instance is expensive, we should reuse

Are you sure about that? Maybe refresh after 500 tenants. Seeding tenants takes 300ms+ after the first few thousand. Opening dbContext should take <10ms.

I suggest creating a testdb with 20k tenants and running some tests on the migrator.

> Repository is abstract, you can do it in your project. see : https://github.com/abpframework/abp/issues/9652

Surely I can't disabled change tracking for ABP seed code though, not sure that would be safe even if I could. We do not yet have our own seeding.

liangshiwei created 3 years ago

Support Team Fullstack Developer

Hi,

I have test, you can try SeedDataAsync method in the <YourProjectName>DbMigrationService and replace it with the following code:

private async Task SeedDataAsync(Tenant tenant = null)
{
    Logger.LogInformation($"Executing {(tenant == null ? "host" : tenant.Name + " tenant")} database seed...");

    using (var uow = _unitOfWorkManager.Begin(true, true))
    {
        await _dataSeeder.SeedAsync(new DataSeedContext(tenant?.Id)
            .WithProperty(IdentityDataSeedContributor.AdminEmailPropertyName, IdentityDataSeedContributor.AdminEmailDefaultValue)
            .WithProperty(IdentityDataSeedContributor.AdminPasswordPropertyName, IdentityDataSeedContributor.AdminPasswordDefaultValue)
        );

        await uow.CompleteAsync();
    }
}

Have an answer to this question? Log in and write your answer.