Background bobs no longer run after upgrade from version 6 to version 7. Neither when upgrading to version 8 #6618 | Support

Tony_Albutt created 9 months ago

ABP Framework version: upgraded from v6.0.3 to 7.4.2
UI Type: MVC
Database System: EF Core (SQL Server)
**Tiered (for MVC) no
Exception message and full stack trace: None - no errors in the log files
Steps to reproduce the issue: Background jobs not been triggered after upgrading to version 7.4.2

When I upgraded form version 6.0.3 to 7.4.2, my background jobs are no longer been run None of the 20 ExecuteAsync methods are been triggered. I have 3 user modules with background Jobs. I removed the three modules to see if this may cause issues with the version 7 and 8 updates. This did not resolve any issues.

No log errors and no warnings logged.

This problem is on the development environment and the Live test server.

Has there been any changes, or encasements, for the default implementation for Background Jobs? None of my 20 background jobs are been run after the upgrade. The ExecuteAsync methods are never called.

From the database table AbpBackgroundJobs, none of the LastTryTime fields are populated,0 nor the TryCounts are updated.

JobName = RequerstToBook JobArgs = (Correct json data has been filled) TryCount =0 NextTryTime = correct data and time (earlier today) LastTryTime = NULL IsAbandoned = 0 Priority = 15

This one of my job arguments

    [BackgroundJobName("RequerstToBook")]
    public class RequestToBookArgs
    {
        public string TenantName { get; set; }
        public Guid TenantId { get; set; }
        public Guid UserId { get; set; }
        public string UserName { get; set; }
        public Guid TravelRequestId { get; set; }
        public Guid SupplierId { get; set; }
        public Guid SupplierContactId { get; set; }
        public Guid QuoteId { get; set; }
        public List<Guid> LineItems { get; set; }
        public string NoteToTMC { get; set; }
    }

and this is the background job

public class RequestToBookJob : AsyncBackgroundJob<RequestToBookArgs>, ITransientDependency
{
    private readonly IRequestForQuotesAppService _requestForQuotesAppService;
    private readonly IEmailSender _emailSender;
    private readonly ICurrentTenant _currentTenant;
    private readonly IMailBotAppService _mailBotAppService;

    public RequestToBookJob(
        IEmailSender emailSender,
        ICurrentTenant currentTenant,
        IMailBotAppService mailBotAppService,
        IRequestForQuotesAppService requestForQuotesAppService)
    {
        _emailSender = emailSender;
        _currentTenant = currentTenant;
        _mailBotAppService = mailBotAppService;
        _requestForQuotesAppService = requestForQuotesAppService;
    }

    public override async Task ExecuteAsync(RequestToBookArgs args)
    {
        var watch = new System.Diagnostics.Stopwatch();
        watch.Start();
        var Comment = await ProcessRequestToBookAsync(args);
        watch.Stop();
    }

    private async Task<string> ProcessRequestToBookAsync(RequestToBookArgs args)
    {
        var result = await _requestForQuotesAppService.GenerateRequestToBook(args);
        return result;
    }
}

I have now added a AbpBackgroundJobWorkerOptions to me ApplicationModule. this did not resolve the issue

            Configure<AbpBackgroundJobWorkerOptions>(options =>
            {
                options.DefaultTimeout = 3600; 
            });

in ApplicationNameTestBaseModule : AbpModule

    public override void ConfigureServices(ServiceConfigurationContext context)
    {
        Configure<AbpBackgroundJobOptions>(options =>
        {
            options.IsJobExecutionEnabled = false;
        });

        context.Services.AddAlwaysAllowAuthorization();
    }

I have now updated to version 8.0.2 to see if this fixes the issue, and still no change.

I then created a ne app and added one Background job for testing. BackgroundsJods run in this app.

I compared all library referenced and don't see ant differences between the two applications. Core and new code associated to background Jobs.

How can I see if the background service is querying the database?

How can I resolve this issue?

Thanks and regards Tony

8 Answer(s)

0

liangshiwei created 9 months ago
Support Team Fullstack Developer

Hi,

As I know, We did not change the background jobs system

You can use the source link to debug the BackgroundJobWorker: https://github.com/abpframework/abp/blob/dev/framework/src/Volo.Abp.BackgroundJobs/Volo/Abp/BackgroundJobs/BackgroundJobWorker.cs#L42

https://devblogs.microsoft.com/dotnet/improving-debug-time-productivity-with-source-link/

If you can share a simple project with me, I will check it. my email is shiwei.liang@volosoft.com
0

Tony_Albutt created 9 months ago

Thanks liangshiwei

I will look at the github link and se what I can do.

Thanks and regards Tony
0

Tony_Albutt created 9 months ago

Hi

I have enabled NuGet.org Symbol. I have disabled Disable Just My Code for debugging. I have verified Source Link support is checked.

Cool 👍 Now how do I get Volo.Abp.BackgroundJobs to run, so that I can debug?

Thanks and regards Tony

liangshiwei created 9 months ago

Support Team Fullstack Developer

Hi,

There is an easier way to debug, you can try adding MyBackgroundJobWorker to the domain module.

MyBackgroundJobWorker will try to get the jobs waiting to be executed and execute them

public class MyBackgroundJobWorker : AsyncPeriodicBackgroundWorkerBase
{
    protected const string DistributedLockName = "AbpBackgroundJobWorker";

    protected AbpBackgroundJobOptions JobOptions { get; }

    protected AbpBackgroundJobWorkerOptions WorkerOptions { get; }

    protected IAbpDistributedLock DistributedLock { get; }

    public MyBackgroundJobWorker(
        AbpAsyncTimer timer,
        IOptions<AbpBackgroundJobOptions> jobOptions,
        IOptions<AbpBackgroundJobWorkerOptions> workerOptions,
        IServiceScopeFactory serviceScopeFactory,
        IAbpDistributedLock distributedLock)
        : base(
            timer,
            serviceScopeFactory)
    {
        DistributedLock = distributedLock;
        WorkerOptions = workerOptions.Value;
        JobOptions = jobOptions.Value;
        Timer.Period = WorkerOptions.JobPollPeriod;
    }

    protected override async Task DoWorkAsync(PeriodicBackgroundWorkerContext workerContext)
    {
        await using (var handler = await DistributedLock.TryAcquireAsync(DistributedLockName, cancellationToken: StoppingToken))
        {
            if (handler != null)
            {
                var store = workerContext.ServiceProvider.GetRequiredService<IBackgroundJobStore>();

                var waitingJobs = await store.GetWaitingJobsAsync(WorkerOptions.MaxJobFetchCount);

                if (!waitingJobs.Any())
                {
                    return;
                }

                var jobExecuter = workerContext.ServiceProvider.GetRequiredService<IBackgroundJobExecuter>();
                var clock = workerContext.ServiceProvider.GetRequiredService<IClock>();
                var serializer = workerContext.ServiceProvider.GetRequiredService<IBackgroundJobSerializer>();

                foreach (var jobInfo in waitingJobs)
                {
                    jobInfo.TryCount++;
                    jobInfo.LastTryTime = clock.Now;

                    try
                    {
                        var jobConfiguration = JobOptions.GetJob(jobInfo.JobName);
                        var jobArgs = serializer.Deserialize(jobInfo.JobArgs, jobConfiguration.ArgsType);
                        var context = new JobExecutionContext(
                            workerContext.ServiceProvider,
                            jobConfiguration.JobType,
                            jobArgs,
                            workerContext.CancellationToken);

                        try
                        {
                            await jobExecuter.ExecuteAsync(context);

                            await store.DeleteAsync(jobInfo.Id);
                        }
                        catch (BackgroundJobExecutionException)
                        {
                            var nextTryTime = CalculateNextTryTime(jobInfo, clock);

                            if (nextTryTime.HasValue)
                            {
                                jobInfo.NextTryTime = nextTryTime.Value;
                            }
                            else
                            {
                                jobInfo.IsAbandoned = true;
                            }

                            await TryUpdateAsync(store, jobInfo);
                        }
                    }
                    catch (Exception ex)
                    {
                        Logger.LogException(ex);
                        jobInfo.IsAbandoned = true;
                        await TryUpdateAsync(store, jobInfo);
                    }
                }
            }
            else
            {
                try
                {
                    await Task.Delay(WorkerOptions.JobPollPeriod * 12, StoppingToken);
                }
                catch (TaskCanceledException) { }
            }
        }
    }

    protected virtual async Task TryUpdateAsync(IBackgroundJobStore store, BackgroundJobInfo jobInfo)
    {
        try
        {
            await store.UpdateAsync(jobInfo);
        }
        catch (Exception updateEx)
        {
            Logger.LogException(updateEx);
        }
    }

    protected virtual DateTime? CalculateNextTryTime(BackgroundJobInfo jobInfo, IClock clock)
    {
        var nextWaitDuration = WorkerOptions.DefaultFirstWaitDuration *
                               (Math.Pow(WorkerOptions.DefaultWaitFactor, jobInfo.TryCount - 1));
        var nextTryDate = jobInfo.LastTryTime?.AddSeconds(nextWaitDuration) ??
                          clock.Now.AddSeconds(nextWaitDuration);

        if (nextTryDate.Subtract(jobInfo.CreationTime).TotalSeconds > WorkerOptions.DefaultTimeout)
        {
            return null;
        }

        return nextTryDate;
    }
}

public override async Task OnApplicationInitializationAsync(ApplicationInitializationContext context)
{
    if (context.ServiceProvider.GetRequiredService<IOptions<AbpBackgroundJobOptions>>().Value.IsJobExecutionEnabled)
    {
        await context.AddBackgroundWorkerAsync<MyBackgroundJobWorker>();
    }
}

0

Tony_Albutt created 9 months ago

Thanks

I will add a BackgroundJobWorker to the domain module.

I will let you know how it goes 👍

Regards Tony
0

liangshiwei created 9 months ago
Support Team Fullstack Developer

Hi,

Ok.
0
Tony_Albutt created 9 months ago
Hi

I get IsJobExecutionEnable = false

?context.ServiceProvider.GetRequiredService<IOptions<AbpBackgroundJobOptions>>().Value.IsJobExecutionEnabled false

The only place that options.IsJobExecutionEnabled = false ; is in the DbMigratorModule

I changed that to true, which did not remedy the problem

So to fix this, I added the following to my ApplicationModule

Configure<AbpBackgroundJobOptions>(options => { options.IsJobExecutionEnabled = true; });

Now the backgroundJobs jobs are running

Somehow the default is disabled, when upgrading to version 7 and 8

It would be best to note this, or investigate further.

Best regards Tony
0

liangshiwei created 9 months ago
Support Team Fullstack Developer

Hi,

This is by design, Background Jobs should not be run during the migration process.

But glad you solved the problem

1

Have an answer to this question? Log in and write your answer.