Issue After Many Locations and User Organization mapping #9941

viswajwalith created 20 days ago

We are facing the issue during login process. when we have number of locations "AbpUserOrganizationUnits" (we have around 1500 Locations) and all of he locations is mapped to the users "AbpUserOrganizationUnits", The number of users are around 3000 this was working ok in our previous ABP 5 solution but after upgrading to Abp9 we found lot time it is taking top process the below code, around 3 minutes hence getting the timeout as well

In order top check this We have created a simple layered solution, and loaded with 2000 Locations. in this case we observed lo of time is taking at

As per our analysis we are suspecting on claims. This is very critial and makor concern.

AI-Bot created 20 days ago

Support Team ABP Agent

You are experiencing severe performance issues during login when users are mapped to a large number of organization units (e.g., 1500+ per user, 3000 users total) after upgrading to ABP 9.0. This is a known challenge when the claims principal is populated with a very large number of organization unit claims, as each claim is loaded and processed during authentication.

Solution & Recommendations

Claims Optimization:
- By default, ABP adds all organization unit IDs as claims to the user during login. With thousands of claims, this can cause significant performance degradation.
- You should avoid loading all organization unit claims into the user's principal if not strictly necessary for every request.
Recommended Approaches:
- Reduce Claims Count: Only add essential claims to the principal. If you only need to check organization membership occasionally, consider querying the database or using a cache instead of claims.
- Custom Claims Principal Factory: Override the IUserClaimsPrincipalFactory to limit or filter the organization unit claims added to the principal.
- Use Data Filters: For data access, use ABP's data filtering and permission system instead of relying on claims for every organization unit.
- Paging or Caching: If you must check many organization units, implement paging or caching strategies outside the claims system.
Reference Implementation:
- See the official documentation for organization units and claims:
  - ABP Identity Module: Organization Units
- See a similar performance issue and solution discussion:
  - ABP Support: Register + Login slow with many tenants/claims
  - ABP Support: Identity Server running slow
Example: Custom Claims Principal Factory

public class MyUserClaimsPrincipalFactory : AbpUserClaimsPrincipalFactory
   {
       public MyUserClaimsPrincipalFactory(
           UserManager<IdentityUser> userManager,
           RoleManager<IdentityRole> roleManager,
           IOptions<IdentityOptions> optionsAccessor)
           : base(userManager, roleManager, optionsAccessor)
       {
       }

       protected override async Task AddOrganizationUnitsAsync(IdentityUser user, ClaimsIdentity identity)
       {
           // Only add a limited number of OUs, or none, or a summary claim
           // Example: Add only the first 10 OUs
           var organizationUnits = await UserManager.GetOrganizationUnitsAsync(user);
           foreach (var ou in organizationUnits.Take(10))
           {
               identity.AddClaim(new Claim(AbpOrganizationUnitClaimType, ou.Id.ToString()));
           }
       }
   }

ABP 9.x Changes:
- ABP 9 introduced some changes in claims and dynamic claims handling. If you have custom logic, review the upgrade notes and ensure your customizations are compatible.

Summary

Do not add thousands of claims to the user principal.
Use custom claims principal logic to limit claims.
Use ABP's data filtering and permission system for organization unit checks.
Review and optimize your login and claims logic for large-scale organization unit scenarios.

Sources:

This response was generated by AI. If it didn’t solve your problem, feel free to post a new answer — our support team will jump in to help you soon.

maliming created 20 days ago

Support Team Fullstack Developer

Can you share your test project?

liming.ma@volosoft.com

Thanks.

viswajwalith created 20 days ago

[maliming] said: hi

Can you share your test project?

liming.ma@volosoft.com

Thanks.

Hi, You can give a try by creating a sample layered solution and creating the Locations

I emailed the script...

Let me know in case of any queries....

maliming created 17 days ago

Support Team Fullstack Developer

Can you share some code to seed organzation data? Then reproduce it?

Thanks.

viswajwalith created 17 days ago

[maliming] said: hi

Can you share some code to seed organzation data? Then reproduce it?

Thanks.

Hi I sent a email with sthe script to pre-seed the data.

maliming created 17 days ago

Support Team Fullstack Developer

The SQL script doesn't reproduce the problem.

Can you try to add your test organizations by code?

Thanks.

viswajwalith created 17 days ago

[maliming] said: hi

The SQL script doesn't reproduce the problem.

Can you try to add your test organizations by code?

Thanks.

You mean even after adding around 4K locations issue not replicated?

maliming created 17 days ago

Support Team Fullstack Developer

I mean, can you add some code to insert organizations to show the problem?

Thanks.

viswajwalith created 17 days ago

[maliming] said: hi

I mean, can you add some code to insert organizations to show the problem?

Thanks.

are u not getting the issue after seeding around 4k Locations?

maliming created 17 days ago

Support Team Fullstack Developer

I am still unable to reproduce the problem.

Can you share such a project?

Thanks.

viswajwalith created 17 days ago

[maliming] said: hi

I am still unable to reproduce the problem.

Can you share such a project?

Thanks.

Ok I will share the project and DB back up as well (in some time)

viswajwalith created 17 days ago

[viswajwalith] said:

[maliming] said: hi

I am still unable to reproduce the problem.

Can you share such a project?

Thanks.

Ok I will share the project and DB back up as well (in some time)

Hi I have sent the files to your email. Issue is coming after mapping the user to locations in "AbpUserOrganizationUnits". Please have a look and let us know if you need any more details.

maliming created 17 days ago

Support Team Fullstack Developer

Thanks. I will download and check it.

👍

maliming created 17 days ago

Support Team Fullstack Developer

Can you try this?

public override void ConfigureServices(ServiceConfigurationContext context)
{
    Configure<AbpEntityOptions>(options =>
    {
        options.Entity<IdentityUser>(identityUserOptions =>
        {
            identityUserOptions.DefaultWithDetailsFunc = queryable =>
            {
                return queryable
                    .Include(x => x.Roles)
                    .Include(x => x.Logins)
                    .Include(x => x.Claims)
                    .Include(x => x.Tokens);
                //.Include(x => x.OrganizationUnits);
            };
        });
    });
}

viswajwalith created 17 days ago

[maliming] said: Configure<AbpEntityOptions>(options => { options.Entity<IdentityUser>(identityUserOptions => { identityUserOptions.DefaultWithDetailsFunc = queryable => { return queryable .Include(x => x.Roles) .Include(x => x.Logins) .Include(x => x.Claims) .Include(x => x.Tokens); //.Include(x => x.OrganizationUnits); }; }); });

Nope still issue persists, did u executed the SQL script to seed the data ? as of now home many entries are there in AbpUsers and AbpUserOrganizationUnits table?.

If needed I can share the screen for better understanding

maliming created 17 days ago

Support Team Fullstack Developer

Can you also override the EfCoreIdentityUserRepository?

The main problem is that user has too many organizations. then .Include(x => x.OrganizationUnits) will take long time.

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Volo.Abp.DependencyInjection;
using Volo.Abp.EntityFrameworkCore;
using Volo.Abp.Identity;
using Volo.Abp.Identity.EntityFrameworkCore;

namespace TestUsersLoad.EntityFrameworkCore;

[Dependency(ReplaceServices = true)]
[ExposeServices(typeof(EfCoreIdentityUserRepository), typeof(IIdentityUserRepository))]
public class MyEfCoreIdentityUserRepository : EfCoreIdentityUserRepository
{
    public MyEfCoreIdentityUserRepository(IDbContextProvider<IIdentityDbContext> dbContextProvider) : base(dbContextProvider)
    {
    }

    public override async Task<IdentityUser?> FindByNormalizedUserNameAsync(
        string normalizedUserName,
        bool includeDetails = true,
        CancellationToken cancellationToken = default)
    {
        return await (MyIncludeDetails(await GetDbSetAsync(), includeDetails))
            .OrderBy(x => x.Id)
            .FirstOrDefaultAsync(
                u => u.NormalizedUserName == normalizedUserName,
                GetCancellationToken(cancellationToken)
            );
    }

    private static IQueryable<IdentityUser> MyIncludeDetails(IQueryable<IdentityUser> queryable, bool include = true)
    {
        if (!include)
        {
            return queryable;
        }

        return queryable
            .Include(x => x.Roles)
            .Include(x => x.Logins)
            .Include(x => x.Claims)
            .Include(x => x.Tokens);
        //.Include(x => x.OrganizationUnits);
    }
}

viswajwalith created 17 days ago

[maliming] said: private static IQueryable<IdentityUser> MyIncludeDetails(IQueryable<IdentityUser> queryable, bool include = true) { if (!include) { return queryable; }
    return queryable 
        .Include(x => x.Roles) 
        .Include(x => x.Logins) 
        .Include(x => x.Claims) 
        .Include(x => x.Tokens); 
    //.Include(x => x.OrganizationUnits); 
} 

Even after overing same issue. the request got struck at

The ReturnUrl is /connect/authorize?client_id=TestUsersLoad_Web&redirect_uri=https%3A%2F%2Flocalhost%3A44348%2Fsignin-oidc&response_type=code%20id_token&scope=openid%20profile%20roles%20email%20phone%20TestUsersLoad&response_mode=form_post&nonce=638947468557304847.N2IyODhhMWQtNzM0Ny00Zjk4LWJhMWUtYjJiMTA4NDU5MDBhYzE2NmM1NWMtM2U4Ni00NWI3LThmOTMtMzNhNWE5Y2YxZTJm&state=CfDJ8GwxaJ_oChhEifGznYgZ_6Hc3q98FIXQjmCCyU_9IxjO0d7a6H5Pc3q8CUh37y_6tEuJDU3MLm8jP7fSbyRiIuTDGJMEd2AI3I-b_mb9-mUnWICbzRduRoarPufcVLWYMjeSRLmvmXTGUhgbSbdAdhXyqzeu309JIRf0I18swYUmfJgZzH_56uPQjFBlb7SCQtuV-Yi3YdhlmLJaE5ysts7L6tUKJOle7CZ0FTcdOTFFymmjBBD-lqlg4SVxR5MRTsIj2WI3hlGrbmODT4qfWy1RPVJ19a-aIq0Z094DCkpE&x-client-SKU=ID_NET9_0&x-client-ver=8.9.0.0

I made small change at ur code as the intended code is not reachable.

maliming created 16 days ago

Support Team Fullstack Developer

You can send an email to liming.ma@volosoft.com

I will share the Zoom link. Please share your screen.

Thanks.

viswajwalith created 16 days ago

[maliming] said: hi

You can send an email to liming.ma@volosoft.com

I will share the Zoom link. Please share your screen.

Thanks.

I have shared the log files

maliming created 16 days ago

Support Team Fullstack Developer

Thanks. I will check them.

maliming created 16 days ago

Support Team Fullstack Developer

The logs.txt doesn't contain the EF Core logs.

Can you upload your project to a private GitHub repository and invite https://github.com/maliming?

I will check and update the code.

The screenshot from my test app:

Thanks.

viswajwalith created 16 days ago

[maliming] said: hi

The logs.txt doesn't contain the EF Core logs.

Can you upload your project to a private GitHub repository and invite https://github.com/maliming?

I will check and update the code.

The screenshot from my test app:

Thanks.

added to private repo and sent the invite to ur email

maliming created 16 days ago

Support Team Fullstack Developer

I added the new code https://github.com/Exceego-Info-Labs-Pvt-Ltd/POC/commit/7343de0961e355aa0c75f44913ecadec8342f0db

Can you delete logs.txt, reproduce the login problem, and share the logs.txt again?

Thanks.

viswajwalith created 16 days ago

[maliming] said: hi

I added the new code https://github.com/Exceego-Info-Labs-Pvt-Ltd/POC/commit/7343de0961e355aa0c75f44913ecadec8342f0db

Can you delete logs.txt, reproduce the login problem, and share the logs.txt again?

Thanks.

I have emailed the updated logs to your email

maliming created 16 days ago

Support Team Fullstack Developer

Can you test this commit?

https://github.com/Exceego-Info-Labs-Pvt-Ltd/POC/commit/2626cb40fb32a5b9c8482609c587a59492132d49

Thanks.

Question details

ABP Version (vX.X.X)

Database Management System

Tiered (MVC) or Auth Server Separated (Angular)

Boost Your Development

ABP Live Training
Packages

See Trainings

The Official Guide

Mastering
ABP Framework

Learn More

Framework

Tools

Startup Templates

UI Themes

Mobile

Modules

39 Answer(s)

Solution & Recommendations

Summary

ABP Live Training
Packages

Mastering
ABP Framework

Framework

Tools

Startup Templates

UI Themes

Mobile

Modules

More

More

More

More

Issue After Many Locations and User Organization mapping #9941

39 Answer(s)

Solution & Recommendations

Summary

ABP Live TrainingPackages

MasteringABP Framework

ABP Live Training
Packages

Mastering
ABP Framework