Closed

redis commands timeout #5847

portx-dev created 2 years ago

Hi, we have redis commands that timeout. I asked that problem in StackExchange/StackExchange.Redis, and got the answer.

It looks like you're experiencing thread exhaustion (WORKER: (Busy=38,Free=32729,Min=8,Max=32767)), likely from long-running synchronous operations. I can't say about the rest of your application, but you for sure have some synchronous Redis calls (sync-ops: 7262, async-ops: 28127). You'll want to do all those things async to have much better utilization of the thread pool and less blocking/stalls as a result of thread exhaustion (waiting for a new one to become available).

reference: https://github.com/StackExchange/StackExchange.Redis/issues/2558

According to the analysis, what we can do for solving timeout issue in ABP Framework?

ABP Framework version: Commercial 7.2 UI type: Angular DB provider: EF Core

5 Answer(s)

0

Anjali_Musmade created 2 years ago
Support Team Member

Hello https://support.aspnetzero.com/QA/Questions/6559/StackExchangeRedisRedisTimeoutException-Timeout#answer-b6aac0d5-f5e1-145e-cf82-39ecb0563c4d Have you tried this syncTimeout
0

portx-dev created 2 years ago

Thanks for sharing that. But increasing the timeout is not fundamental solution. The problem is slow commands. I doubt many redis commands dispatched at the same time affect to bad performance. One of the requests dispatched1456 redis commands. Is that normal?
0

Anjali_Musmade created 2 years ago
Support Team Member

Hello portx-dev,

Actually this is a performance issue try to optimize your queries and code. For Worker Busy Thread < Min Thread

I am providing some workaround please try this - You need to change your Min Thread for that please refer this https://learn.microsoft.com/en-us/dotnet/core/runtime-config/threading or try to add this code in Program.cs of Host

System.Threading.ThreadPool.SetMinThreads(200, 200);

I have given here Min Thread value as 200 you can change it as per your requirement

Please do let me know if this helps you

Thank you, Anjali
0

portx-dev created 2 years ago

Thanks for providing solution. The performance was improved by increasing min threads.

On the other side, there are still requests that contain many redis command. I think 200~ commands in a request is irregular number, isn't it?
0

Anjali_Musmade created 2 years ago
Support Team Member

Hello portx-dev,

In one of our similar deployed application we are using Min Thread = 1000 and its working fine.

Regards,