Open Closed

Azure Distributed Events Not Always Working #6222


User avatar
0

Working on using distributed events, though what we're seeing is that not all events are being received as expected (half?):

What you're seeing:

  • We're triggering a background job to be run from Web
  • That background job is being run within the Api.Host
  • When the background job is complete, we publish an event to notify the client (for the purpose of client-side notification)
  • In some cases (approximately half?) the message is received and can be dealt with properly (in SignalR hub which then causes client-side handler to ultimately call abp.notify
  • In other cases, nothing is occurring.

  • ABP Framework version: v7.4.2
  • UI Type: MVC
  • Database System: EF Core (SQL Server)
  • Tiered (for MVC) or Auth Server Separated (for Angular): yes
  • Steps to reproduce the issue:

Here's the options:

Configure<AbpAzureServiceBusOptions>(options =>
{
    options.Connections.Default.Admin.Retry.MaxRetries = 3;
    options.Connections.Default.Client.RetryOptions.Delay = TimeSpan.FromMilliseconds(100);
    options.Connections.Default.Client.RetryOptions.MaxDelay = TimeSpan.FromSeconds(10);
    options.Connections.Default.Client.RetryOptions.Mode = Azure.Messaging.ServiceBus.ServiceBusRetryMode.Exponential;
    options.Connections.Default.Client.RetryOptions.MaxRetries = 10;
});

11 Answer(s)
  • User Avatar
    0

    There's a repro here: https://github.com/victoriatolls/Acme.BookStore.DistributedEvent

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    Thanks. I will check it asap.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    Please make your repo private, I have downloaded the source code.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    Please share an azure token, I will test it locally. Thanks.

  • User Avatar
    0

    Will share with liming.ma@volosoft.com

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    Thanks. Can you invite me (https://github.com/maliming) to your Acme.BookStore.DistributedEvent repo?

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

     await _backgroundJobManager.EnqueueAsync<TestJobArgs>(new TestJobArgs() { Content = content, UserId = CurrentUser.Id, TenantId = CurrentUser.TenantId });
    

    This operator will store a job to the database, and the BackgroundJobWorker will check the job and execute it every 5 seconds.

    It's not executed in real-time.

    https://github.com/abpframework/abp/blob/72681ab6264b8ef5fb4878ed5cd0caedb99044b1/framework/src/Volo.Abp.BackgroundJobs/Volo/Abp/BackgroundJobs/AbpBackgroundJobWorkerOptions.cs#L9

  • User Avatar
    0

    Yea, we understand that - that's by design. While we know the background job runs when it runs, the job itself pubs the distributed event - why doesn't the event get received half the time?

    Ultimately, this is all just to notify a user when a job completes independent of the page they are on (as reporting jobs can take long) - if there's a better way, we're all ears - but there isn't AFAIK currently and this should work.

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    I will check it again.

  • User Avatar
    1
    maliming created
    Support Team Fullstack Developer

    hi

    The solution is to set IsServiceBusDisabled = true in HTTP API module.

  • User Avatar
    0

    hi

    The solution is to set IsServiceBusDisabled = true in HTTP API module.

    Yes, that seems to have done the trick. At the same time, for whatever reason, we also had to produce new keys. For some reason the default key that's created when you create the service bus doesn't work while a new key (created to be the same) works. Even scoped keys work (scoped to a topic), they also work.

    Also, the "manage" permission is required or this'll be what happens:

Made with ❤️ on ABP v9.1.0-preview. Updated on December 05, 2024, 12:19