ABP Commercial 9.1.0 / Blazor Server / EF / Non tiered / Separate Host and Tenant DBs / Lepton Theme
Hi, We have had customers experience intermittent occurrences of receiving a duplicate email through our ABP app.
After investigation, we found that this is caused by our production app being scaled out to 2 application instances in an Azure Appservice.
Steps:
- Instance A serves the request to send an email
- The email gets queued in the AbpBackgroundJobs table in the tenant host database
- Instance A and B see the queued email in the AbpBackgroundJobs table, causing a potential race condition that results in both instances sending the email
Please advise on the best approach to handle this scenario using the ABP Framework.
Thanks in advance!
2 Answer(s)
-
0
What you are seeing is expected with ABP’s default background job executor in a scaled-out (multiple instance) environment: each instance runs its own background job worker, so if the storage/provider doesn’t guarantee distributed “pick and lock”, you can end up with duplicate executions.
Best practice options (recommended order):
Use a real distributed background job provider (Hangfire)
- Configure ABP Background Jobs to use Hangfire, so job acquisition/execution is coordinated across instances (Hangfire uses distributed locks in its storage).
- This is the recommended approach when you want multiple servers/instances processing jobs reliably and you want retries, dashboards, etc.
- ABP already has integration so your app code that enqueues jobs doesn’t need to change.
Run background job execution in a separate “job executor” application (single instance)
- Keep your web app scaled out, but disable job execution in the web app(s), and run a dedicated console/worker app that executes jobs (can be scaled independently later with a proper provider).
- In ABP, you can disable job execution in the web application via:
Configure<AbpBackgroundJobOptions>(options => { options.IsJobExecutionEnabled = false; });- Then create a separate console app that references your job types and runs the background job module/provider.- Add idempotency at the business level (still recommended even with 1/2)
- For “send email” jobs, store a unique business key (e.g.,
EmailNotificationId/CorrelationId) and enforce “send only once” in your own tables (unique index + “already sent” check). - This protects you not only from scale-out races, but also from retries, transient failures, and user double-clicks.
- For “send email” jobs, store a unique business key (e.g.,
- Practical guidance for your exact scenario (Azure App Service, 2 instances):
- If you want to keep background jobs inside the same web host(s), switch to Hangfire integration (Option 1).
- If you want the simplest “guarantee single execution” without bringing a distributed job system immediately, implement Option 2 (single executor) and later you can still move that executor to Hangfire if you need throughput.
Links (validated):
- https://abp.io/docs/latest/deployment/optimizing-production
- https://abp.io/docs/latest/framework/infrastructure/background-jobs
- https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire
- https://abp.io/community/articles/my9cgo9a
- https://github.com/abpframework/abp-samples/tree/master/SeparateBackgroundJob
This response was generated by AI. If it didn’t solve your problem, feel free to post a new answer — our support team will jump in to help you soon.
-
0
hi
You can set only one app to execute the job, or set an application name for isolation(>= 9.2)
See: https://abp.io/docs/latest/framework/infrastructure/background-jobs#disable-job-execution https://abp.io/docs/latest/framework/infrastructure/background-jobs#using-the-same-storage-for-background-jobs--workers
Thanks.