Open Closed

Migrations EFcore loop issue #1807


User avatar
0
dmeagor created
  • ABP Framework version: Any
  • UI type: Any
  • DB provider: EF Core
  • Tiered (MVC) or Identity Server Separated (Angular): Any

I have previously reported this as a bug here. https://github.com/abpframework/abp/issues/9467

ABP EFCore migrations are currently running N+1 loops with the tenant seeding without recreating the dbContext instance for each loop. This is against microsoft's EFCore guidence and causes severe slowdowns after a few hundred tenants. I believe this is due to change tracking.

This seems fairly simple to fix but it's not something we can do here. Possible fixes are.

  • Don't do long iterations. Pull out permissions etc. in one query. Looking at PermissionDataSeeder.cs this seems like a quick change.
  • Recreate the dbContext object inside of the loop for each iteration ( each tenant )
  • Turn off change tracking for the dbContext instance or specific queries.

The quick fixes seem to be disabling change tracking or recreating the dbContext on each tenant loop. I also think pulling out the permissions in bulk rather than one at a time would make a big difference.

For our number of users is it entirelly impossible to migrate the database seeding (structural changes are fast though.)

I submitted this to github and to bugs and issues and got no response, just closed as stale.


4 Answer(s)
  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Hi,

    I will check it out.

  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Don't do long iterations. Pull out permissions etc. in one query. Looking at PermissionDataSeeder.cs this seems like a quick change.

    I will try impove it, see https://github.com/abpframework/abp/pull/9978

    Recreate the dbContext object inside of the loop for each iteration ( each tenant )

    Database context instance is expensive, we should reuse

    Turn off change tracking for the dbContext instance or specific queries.

    Repository is abstract, you can do it in your project. see : https://github.com/abpframework/abp/issues/9652

  • User Avatar
    0
    dmeagor created

    Thanks for your help with this!

    Recreate the dbContext object inside of the loop for each iteration ( each tenant )

    Database context instance is expensive, we should reuse

    Are you sure about that? Maybe refresh after 500 tenants. Seeding tenants takes 300ms+ after the first few thousand. Opening dbContext should take <10ms.

    I suggest creating a testdb with 20k tenants and running some tests on the migrator.

    > Repository is abstract, you can do it in your project. see : https://github.com/abpframework/abp/issues/9652

    Surely I can't disabled change tracking for ABP seed code though, not sure that would be safe even if I could. We do not yet have our own seeding.

  • User Avatar
    0
    liangshiwei created
    Support Team Fullstack Developer

    Hi,

    I have test, you can try SeedDataAsync method in the <YourProjectName>DbMigrationService and replace it with the following code:

    private async Task SeedDataAsync(Tenant tenant = null)
    {
        Logger.LogInformation($"Executing {(tenant == null ? "host" : tenant.Name + " tenant")} database seed...");
    
        using (var uow = _unitOfWorkManager.Begin(true, true))
        {
            await _dataSeeder.SeedAsync(new DataSeedContext(tenant?.Id)
                .WithProperty(IdentityDataSeedContributor.AdminEmailPropertyName, IdentityDataSeedContributor.AdminEmailDefaultValue)
                .WithProperty(IdentityDataSeedContributor.AdminPasswordPropertyName, IdentityDataSeedContributor.AdminPasswordDefaultValue)
            );
    
            await uow.CompleteAsync();
        }
    }
    
Made with ❤️ on ABP v9.1.0-preview. Updated on December 05, 2024, 12:19