Starts in:
2 DAYS
6 HRS
36 MIN
25 SEC
Starts in:
2 D
6 H
36 M
25 S
Open Closed

Database exceptions while running in Azure, how to handle without EnableRetryOnFailure #7848


User avatar
0
okains created
  • ABP Framework version: v8.2.3
  • UI Type: Blazor Server
  • Database System: EF Core (SQL Server)
  • Tiered (for MVC) or Auth Server Separated (for Angular): yes
  • Exception message and full stack trace:
  • Steps to reproduce the issue:

Hi,

I have been deploying to Azure Deployment Slots, and I am getting transient db errors in the logs on the Auth project that are causing me to often get 400 Errors returned. The log errors are initially in this format:

2024-09-06 15:20:30.990 +00:00 [ERR] An error occurred using the connection to database 'ESv2-testing' on server 'tcp:hathor-hk.database.windows.net,1433'.

Resulting in this:

Following the suggestion of ChatGPT, I added EnableRetryOnFailure:

Configure<AbpDbContextOptions>(options => { /* The main point to change your DBMS. * See also ESv2DbContextFactory for EF Core tooling. */ options.UseSqlServer(sqlOptions => sqlOptions.EnableRetryOnFailure(5, TimeSpan.FromSeconds(10), null)); });

But it seems this may not be supported in ABP.

I am now getting the following in my logs :

2024-09-06 13:08:13.988 +00:00 [ERR] An exception occurred while iterating over the results of a query for context type 'Volo.Abp.TextTemplateManagement.EntityFrameworkCore.TextTemplateManagementDbContext'. System.InvalidOperationException: The configured execution strategy 'SqlServerRetryingExecutionStrategy' does not support user-initiated transactions. Use the execution strategy returned by 'DbContext.Database.CreateExecutionStrategy()' to execute all the operations in the transaction as a retriable unit. at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.OnFirstExecution()

Is there another approach to handling these errors?

This particular exception that is logged: [ERR] An exception occurred while iterating over the results of a query for context type 'Volo.Abp.TextTemplateManagement.EntityFrameworkCore.TextTemplateManagementDbContext'.

Is this maybe the root cause here? This doesn't happen all of the time though, any idea on what is going on here?

Thanks,

Karim Ainsworth


34 Answer(s)
  • User Avatar
    0
    okains created

    Hi,

    at ESv2.Pages.IndexModel.OnGetAsync() in D:\a\Hathor\Hathor\aspnet-core\src\ESv2.AuthServer\Pages\Index.cshtml.cs:line 30

    OK that is in the Auth project, the line that I added back in as it didn't have any effect previously on these issues. I will comment this out again and update the CancellationTokenProvider and send logs.

    Thanks,

    Karim

  • User Avatar
    0
    okains created

    Btw You can try using NullCancellationTokenProvider instead HttpContextCancellationTokenProvider to troubleshoot the A task was canceled. problem

     
    public override void ConfigureServices(ServiceConfigurationContext context) 
    { 
        context.Services.Replace(ServiceDescriptor.Transient<ICancellationTokenProvider, NullCancellationTokenProvider>()); 
             
    } 
    

    Getting an error here:

    Any idea on how to get this to work?

    Thanks,

    Karim

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    public class MyNullCancellationTokenProvider : CancellationTokenProviderBase
    {
        public static NullCancellationTokenProvider Instance { get; } = new();
    
        public override CancellationToken Token => OverrideValue?.CancellationToken ?? CancellationToken.None;
    
        public MyNullCancellationTokenProvider()
            : base(new AmbientDataContextAmbientScopeProvider<CancellationTokenOverride>(new AsyncLocalAmbientDataContext()))
        {
        }
    }
    
    
    
  • User Avatar
    0
    okains created

    Hi,

    OK I have commented out the AppService call in Index, and also the db call in Index in the Auth project. I don't see any of those DB errors in the logs now, though still having ABPAuthentication errors ( different support ticket ).

    Here are the new logs:

    https://1drv.ms/u/s!AkJmGuHQuob7kqdJN7sF0etJ-CbLzQ?e=GYNaHp

    Is there a way for me to fix this so that I can add back my code? Is this related to the OpenIDDict db context? Hope to be able to find a proper fix for this so that I can continue working.

    Thanks,

    Karim

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    Try to use the below Index Get code.

    public async Task OnGetAsync()
    {
        var cancellationTokenSource = new CancellationTokenSource();
        var cancellationToken = cancellationTokenSource.Token;
        Applications = await OpenIdApplicationRepository.GetListAsync(cancellationToken: cancellationToken);
    
        Languages = await LanguageProvider.GetLanguagesAsync();
        CurrentLanguage = CultureInfo.CurrentCulture.DisplayName;
        
        cancellationTokenSource.Dispose();
    }
    
  • User Avatar
    0
    okains created

    Hi,

    Sorry for late reply, was travelling. I will add this in and do some further testing, if the db issues are no longer in the logs then that will be great. My only concern is that there is a lot of extra code / configuration to get to this point and that might have other unintended effects. I am going to be doing some heavy load testing on this app next week so I want to make sure we haven't introduced any issues that might come up during testing.

    For now please keep this ticket open until I can finish my testing and confirm that things are OK.

    Thanks,

    Karim

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    ok

  • User Avatar
    0
    okains created

    As discussed, waiting on another support ticket to clear on this then I will run load testing, please keep open just in case these issues occur again during load testing. Once I get through the other support ticket, and the testing, I will close.

    Thanks,

    Karim

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    ok

Made with ❤️ on ABP v9.1.0-preview. Updated on November 20, 2024, 13:06