Open Closed

redis commands timeout #5847


User avatar
0
portx-dev created

Hi, we have redis commands that timeout.
I asked that problem in StackExchange/StackExchange.Redis, and got the answer.

It looks like you're experiencing thread exhaustion (WORKER: (Busy=38,Free=32729,Min=8,Max=32767)), likely from long-running synchronous operations. I can't say about the rest of your application, but you for sure have some synchronous Redis calls (sync-ops: 7262, async-ops: 28127). You'll want to do all those things async to have much better utilization of the thread pool and less blocking/stalls as a result of thread exhaustion (waiting for a new one to become available).

reference: https://github.com/StackExchange/StackExchange.Redis/issues/2558

According to the analysis, what we can do for solving timeout issue in ABP Framework?


ABP Framework version: Commercial 7.2
UI type: Angular
DB provider: EF Core


5 Answer(s)
  • User Avatar
    0
    Anjali_Musmade created
    Support Team Member
  • User Avatar
    0
    portx-dev created

    Thanks for sharing that. But increasing the timeout is not fundamental solution. The problem is slow commands.
    I doubt many redis commands dispatched at the same time affect to bad performance.
    One of the requests dispatched1456 redis commands. Is that normal?
    スクリーンショット 2023-09-27 11.04.39.png

  • User Avatar
    0
    Anjali_Musmade created
    Support Team Member

    Hello portx-dev,

    Actually this is a performance issue try to optimize your queries and code.
    For Worker Busy Thread < Min Thread

    image.png

    I am providing some workaround please try this -
    You need to change your Min Thread for that please refer this https://learn.microsoft.com/en-us/dotnet/core/runtime-config/threading
    or try to add this code in Program.cs of Host

    System.Threading.ThreadPool.SetMinThreads(200, 200);

    I have given here Min Thread value as 200 you can change it as per your requirement

    Please do let me know if this helps you

    Thank you,
    Anjali

  • User Avatar
    0
    portx-dev created

    Thanks for providing solution.
    The performance was improved by increasing min threads.

    On the other side, there are still requests that contain many redis command. I think 200~ commands in a request is irregular number, isn't it?

    スクリーンショット 2023-09-28 13.42.13.png

  • User Avatar
    0
    Anjali_Musmade created
    Support Team Member

    Hello portx-dev,

    In one of our similar deployed application we are using Min Thread = 1000 and its working fine.

    Regards,

Boost Your Development
ABP Live Training
Packages
See Trainings
Mastering ABP Framework Book
Do you need assistance from an ABP expert?
Schedule a Meeting
Mastering ABP Framework Book
The Official Guide
Mastering
ABP Framework
Learn More
Mastering ABP Framework Book
Made with ❤️ on ABP v9.2.0-preview. Updated on March 13, 2025, 04:08