Background Job For Image Processing #10649

5 Answer(s)

• 

  0

  AI-Bot created 5 months ago

  Support Team ABP Agent

  • For “run at night, every day, for any timezone”, use an ABP Background Worker with the Quartz integration (cron scheduling + persistence). A “Background Job” in ABP is mainly for one-time / fire-and-forget work items; your case is a recurring schedule.
  • Make the image processing itself idempotent and chunked:
    • Don’t process the same original image repeatedly: store a “retina generated” flag/version (or hash) in your DB.
    • If you have many images, process in batches/pages to avoid long transactions and memory pressure.
  • Recommended architecture for your scenario:
    1. Store original images in a durable store (e.g., Blob storage / file system) and keep metadata in DB (original size, last-processed time, hash, etc.).
    2. A nightly Quartz worker finds “images needing retina versions” and generates the 2x versions.
    3. Save the retina outputs back to storage and update metadata.
  • Timezone advice (so it works “at night time for any timezone”):
    • Best practice: schedule the worker to run frequently (e.g., every 15–60 minutes) and have it process only tenants/users whose local time is currently within your nightly window (e.g., 01:00–04:00). This avoids trying to create one cron trigger per timezone and keeps scheduling simple and robust.
    • Persist each tenant’s desired timezone (IANA/Windows) and “night window” in your settings table; compute “now in tenant timezone” inside the worker, then decide whether to process.
  • Operational advice before you start:
    • Use persistent Quartz storage (SQL Server) so schedules survive restarts/deployments.
    • Add concurrency protections (e.g., prevent overlapping runs) and consider misfire behavior (what happens if the app was down at the scheduled time).
    • Add logging/metrics around: number of images scanned, processed, failed; duration; retry counts.
    • Keep the actual image resize code in a domain/application service; the worker should orchestrate only.

  Minimal Quartz worker skeleton (ABP style)

  using Quartz;
  using Volo.Abp.BackgroundWorkers.Quartz;
  
  public class RetinaImageNightlyWorker : QuartzBackgroundWorkerBase
  {
      public RetinaImageNightlyWorker()
      {
          JobDetail = JobBuilder.Create<RetinaImageNightlyWorker>()
              .WithIdentity(nameof(RetinaImageNightlyWorker))
              .Build();
  
          // Example: run every 30 minutes (then decide per-tenant “is it night?” inside Execute)
          Trigger = TriggerBuilder.Create()
              .WithIdentity(nameof(RetinaImageNightlyWorker))
              .WithCronSchedule("0 0/30 * * * ?")
              .Build();
  
          ScheduleJob = async scheduler =>
          {
              if (!await scheduler.CheckExists(JobDetail.Key))
              {
                  await scheduler.ScheduleJob(JobDetail, Trigger);
              }
          };
      }
  
      public override async Task Execute(IJobExecutionContext context)
      {
          // Resolve your app/domain service(s) and:
          // 1) iterate tenants
          // 2) if tenant local time is within night window => process a batch of images
      }
  }
  

  Sources

  • https://abp.io/docs/latest/framework/infrastructure/background-workers/quartz
  • https://abp.io/docs/latest/framework/infrastructure/background-workers
  • https://abp.io/community/articles/t98pzjv6
  • https://abp.io/docs/latest/framework/infrastructure/background-jobs
  • https://abp.io/docs/latest/deployment/optimizing-production

  ---

  This response was generated by AI. If it didn’t solve your problem, feel free to post a new answer — our support team will jump in to help you soon.

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel
• 

  0

  punta created 5 months ago

  Great! Let me wait for the Support Teams to have their say

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel
• 

  0

  maliming created 5 months ago

  Support Team Fullstack Developer

  Hi,

  A few things worth knowing before you start, because the design choices here matter more than the code.

  1) Pick the right ABP primitive: Worker, not Job

  ABP has two different things and the names are easy to confuse:

  • IBackgroundJobManager.EnqueueAsync(...) — fire-and-forget, one-time work item. Good for "user just uploaded an image, process this single one in the background."
  • BackgroundWorker — recurring task that runs on a schedule. Good for "scan nightly and process whatever still needs retina versions."

  Your scenario is the second one. You don't need Quartz, Hangfire, or any extra package for this — the default ABP background worker is enough. Quartz integration only becomes useful if you later need cron expressions, persistent schedules, or cluster-wide single-trigger guarantees across multiple worker hosts. For one host running a periodic scan, skip it.

  The two can also be combined: the worker scans for "images needing retina versions" and enqueues a BackgroundJob per image, so multiple jobs can run in parallel and retries are handled per-image. But for a first version, doing the work inline in the worker is fine.

  2) Don't try to express "night in every timezone" as a cron

  The naive instinct is "create one trigger per timezone." Don't. Instead:

  • Run the worker frequently (every 15–30 minutes).
  • For each tenant, compute "what's the local time for this tenant right now?" and only process if it falls inside your night window (e.g. 01:00–04:00).

  ABP already gives you everything you need for this — no need to build your own timezone table:

  • The framework ships a built-in setting TimingSettingNames.TimeZone (key: Abp.Timing.Timezone). Tenant admins can set it from the standard settings UI per tenant.
  • IClock.ConvertToUserTime(DateTime utcNow) converts UTC to the tenant's local time, using ICurrentTimezoneProvider.TimeZone.
  • ICurrentTimezoneProvider.Change(tz) gives you a disposable scope to swap the active timezone inside the worker loop.

  One requirement: configure the clock to use UTC, otherwise SupportsMultipleTimezone returns false and ConvertToUserTime silently returns the input unchanged:

  Configure<AbpClockOptions>(options =>
  {
      options.Kind = DateTimeKind.Utc;
  });
  

  Important — read before flipping this switch. AbpClockOptions.Kind is an application-wide setting, not a worker-local one. The default is Unspecified, which effectively disables the entire ABP Clock/timezone system. Switching to Utc changes how Clock.Now, Clock.Normalize, ConvertToUserTime, and ConvertToUtc behave everywhere in the app — any code that stores, compares, or returns DateTime values is affected. If your app already has data and code written under Unspecified (or Local), audit it before flipping this. Full background, including all the conversion semantics and edge cases:

  https://abp.io/docs/latest/framework/infrastructure/timing

  3) The worker skeleton

  public class RetinaImageNightlyWorker : AsyncPeriodicBackgroundWorkerBase
  {
      public RetinaImageNightlyWorker(
          AbpAsyncTimer timer,
          IServiceScopeFactory serviceScopeFactory)
          : base(timer, serviceScopeFactory)
      {
          Timer.Period = 30 * 60 * 1000; // 30 minutes
      }
  
      protected override async Task DoWorkAsync(PeriodicBackgroundWorkerContext workerContext)
      {
          var tenantStore = workerContext.ServiceProvider.GetRequiredService<ITenantStore>();
          var currentTenant = workerContext.ServiceProvider.GetRequiredService<ICurrentTenant>();
          var timezoneProvider = workerContext.ServiceProvider.GetRequiredService<ICurrentTimezoneProvider>();
          var settingProvider = workerContext.ServiceProvider.GetRequiredService<ISettingProvider>();
          var clock = workerContext.ServiceProvider.GetRequiredService<IClock>();
          var processor = workerContext.ServiceProvider.GetRequiredService<IRetinaImageProcessor>();
  
          foreach (var tenant in await tenantStore.GetListAsync(includeDetails: false))
          {
              using (currentTenant.Change(tenant.Id))
              {
                  var tenantTz = await settingProvider.GetOrNullAsync(TimingSettingNames.TimeZone);
                  if (tenantTz.IsNullOrWhiteSpace()) continue;
  
                  using (timezoneProvider.Change(tenantTz))
                  {
                      var localNow = clock.ConvertToUserTime(DateTime.UtcNow);
                      if (localNow.Hour < 1 || localNow.Hour >= 4) continue; // outside the 01:00-04:00 window
  
                      await processor.ProcessPendingImagesAsync(workerContext.CancellationToken);
                  }
              }
          }
      }
  }
  

  A few things to notice:

  • ICurrentTenant.Change switches the data filter so your repositories see only that tenant's data.
  • ICurrentTimezoneProvider.Change is flow-scoped (AsyncLocal-based), so it's safe inside the loop.
  • Process in batches inside ProcessPendingImagesAsync (e.g. 50 images per pass) so a single pass doesn't run for hours and block the next tick.

  4) Make it idempotent

  Don't reprocess what you've already done. Add a small piece of state on the image record — a RetinaProcessedAt timestamp, a Status enum, or a hash of the source — and let the worker query for "rows where retina is missing or stale." This also handles failures naturally: a failed image stays "pending" and gets retried on the next pass. Cap the retry count so a permanently broken image doesn't loop forever.

  5) Storage

  Keep the original and the retina version in whatever storage you already use. ABP's BlobStoring module is convenient if you don't have one yet (it's tenant-aware out of the box), but anything works — your local DB, S3, Azure Blob, file system. Just store enough metadata to locate both versions.

  6) The image processing itself

  This is where ABP doesn't really opine. "Retina = 2× width and height of the source" is a one-liner in any image library: pick whatever you're comfortable with (a managed library, a native binding, or even shelling out to an external tool). Just keep the resize code behind a thin domain/application service interface (IRetinaImageProcessor above) so the worker stays focused on orchestration and you can swap the underlying library without touching the scheduling logic. One thing to keep in mind: image resize is CPU and memory heavy, so if traffic grows it's worth running the worker in a dedicated host rather than sharing the web app's process.

  7) Things to do up front

  • Set AbpClockOptions.Kind = DateTimeKind.Utc (required for the timezone helpers to work — but this is application-wide, see the warning in section 2 and the timing docs before changing it on an existing app).
  • Make sure each tenant has the Abp.Timing.Timezone setting filled in — otherwise the worker will skip them.
  • Decide on your night window (e.g. 01:00–04:00 local) and your batch size early, since they drive how long a pass takes.
  • If you ever scale to multiple worker hosts, switch to the Quartz integration with a SQL Server jobStore and clustering — that's the only way to guarantee one trigger across the cluster. But on day one, the default worker on a single host is the simpler answer.

  Thanks

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel

  1
• 

  0

  punta created 5 months ago

  Really Great Takeaway! Helpful, Thanks a lot!

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel
• 

  0

  maliming created 5 months ago

  Support Team Fullstack Developer

  👍

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel