Hi,
We found some redis commands are very slow(1~53s), and fails in test environment. We use amazon elasticache(redis) for our test environment.
slow command examples are below.
HMGET t:3a0c1d64-4d96-0ab6-9c31-0a332a2b8a8e,c:Volo.Abp.LanguageManagement.Texts,k:PortX:PortX_ja HMGET t:3a0c1d64-4d96-0ab6-9c31-0a332a2b8a8e,c:Volo.Abp.LanguageManagement.Texts,k:PortX:AbpValidation_en HMGET c:Volo.Saas.Tenants.Tenant,k:PortX:i:3a0c1d64-4d96-0ab6-9c31-0a332a2b8a8e,n:null EXPIRE t:3a0c1d64-4d96-0ab6-9c31-0a332a2b8a8e,c:Volo.Abp.LanguageManagement.Texts,k:PortX:AbpValidation_en EXPIRE t:3a0c1d64-4d96-0ab6-9c31-0a332a2b8a8e,c:Volo.Abp.LanguageManagement.Texts,k:PortX:AbpValidation_ja EXPIRE c:Volo.Saas.Tenants.Tenant,k:PortX:i:3a0c1d64-4d96-0ab6-9c31-0a332a2b8a8e,n:null HMGET t:3a0c1d64-4d96-0ab6-9c31-0a332a2b8a8e,c:Volo.Abp.PermissionManagement.PermissionGrant,k:PortX:pn:R,pk:admin,n:PortX.RegularProjectRequestHistories.Create
We had a lot of time to investigate the reason with Datadog APM and redis insight, but we've not found the exact reason.
[investigation result]
- the most portion of the commands are fast(12.81ms by 99 percentile). few commands are very slow(1~53s).
- server resource is enough (cpu/memory utilization is under 40%)
- redis resource is enough (cpu/memory utilization is under 40%)
- values returned by slow commands are not so large.
- we don't customize framework code which execute redis commands.
- we found the value of the key
t:*:Volo.Abp.LanguageManagement.Texts,k:PortX:PortX_jais almost empty: { "dictionary": {} }
With these slow commands, we couldn't meet our customers needs.
We doubt
- Amazon Elasticache(redis) have bad compatibility with ABP framework?
- We use Amazon Elasticache(redis) with default parameters. There is room for customization?
- Should we customize some configuration in ABP framework?
- We mis config some setting?
ABP Framework version: Commercial 7.2 UI type: Angular DB provider: EF Core
4 Answer(s)
-
0
Hi,
Actually, ABP uses Microsoft's distributed Redis cache package
Can this problem be reproduced with the local Redis server? Could you provide the full steps to reproduce? I will check it.
I have no suggestion if it cannot be reproduced
-
0
I would like to stop using redis to isolate the problem. How can I stop using redis?
-
0
By the way, we use "Amazon ElastiCache for Redis". When using AWS, is there a recommended redis service?
-
0
Hi,
Yes, you can use docker to deploy Redis in the instance.
Actually, Redis is not necessary but the distributed cache.
There are many options:
- Memory distributed cache: Highest performance, but not suitable for distributed or microservice applications
- Redis distributed cache: Good performance
- SqlServer distributed cache: General performance
- NCache distributed cache: Highest performance.
You can check the document: https://learn.microsoft.com/en-us/aspnet/core/performance/caching/distributed?view=aspnetcore-7.0#establish-distributed-caching-services
