How to run ABP background job parallelly #9106

0

berkansasmaz created 16 days ago

Support Team .NET Developer

Hi,

Would you consider to use RabbitMQ or HangFire provider for background jobs? It is possible to process messages in parallel with them. See: https://github.com/abpframework/abp/issues/5217 and https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire

0

priyankasynapxe created 16 days ago

Hi,

Would you consider to use RabbitMQ or HangFire provider for background jobs? It is possible to process messages in parallel with them. See: https://github.com/abpframework/abp/issues/5217 and https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire

Thank you, I'm considering hangfire but I want to understand how smooth will it be to switch from default ABP background job to hangfire, also how hangfire works? e.g. if I get 10 requests from users, so will it process 10 requests at the same time or it will process request picked by each server which means 2 requests at the same time. When I'm processing the background job, I'm calling the external services, and I need to wait for the response to complete the request so with hangfire, will it also wait the existing request to complete, or it will start next request parallelly.

0

berkansasmaz created 15 days ago

Support Team .NET Developer

Hi,

if I get 10 requests from users, so will it process 10 requests at the same time or it will process request picked by each server which means 2 requests at the same time.

This will depend entirely on your configuration. For example:

app.UseHangfireServer(new BackgroundJobServerOptions
{
    WorkerCount = 10 // or any number you need
});

You can configure the degree of parallelism manually like above. See more: https://docs.hangfire.io/en/latest/background-processing/configuring-degree-of-parallelism.html

When I'm processing the background job, I'm calling the external services, and I need to wait for the response to complete the request so with hangfire, will it also wait the existing request to complete, or it will start next request parallelly.

I am not an expert on Hangfire, but when I check their documentation, I see that you can add throttling or concurrency limiters. See: https://docs.hangfire.io/en/latest/background-processing/throttling.html

0

priyankasynapxe created 15 days ago

Hi,

if I get 10 requests from users, so will it process 10 requests at the same time or it will process request picked by each server which means 2 requests at the same time.

This will depend entirely on your configuration. For example:
app.UseHangfireServer(new BackgroundJobServerOptions 
{ 
    WorkerCount = 10 // or any number you need 
}); 
You can configure the degree of parallelism manually like above. See more: https://docs.hangfire.io/en/latest/background-processing/configuring-degree-of-parallelism.html

When I'm processing the background job, I'm calling the external services, and I need to wait for the response to complete the request so with hangfire, will it also wait the existing request to complete, or it will start next request parallelly.

I am not an expert on Hangfire, but when I check their documentation, I see that you can add throttling or concurrency limiters. See: https://docs.hangfire.io/en/latest/background-processing/throttling.html

I'm trying to implement hangfire but as soon as I put Queue attribute, my job is not executing, pls help me to understand if any other step is required, I'm following https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire

0

EngincanV created 14 days ago

Support Team .NET Developer

I'm trying to implement hangfire but as soon as I put Queue attribute, my job is not executing, pls help me to understand if any other step is required, I'm following https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire

Hi, did you configure a storage for Hangfire as follows in your module class?:

      context.Services.AddHangfire(config =>
      {
          config.UseSqlServerStorage(configuration.GetConnectionString("Default"));
      });

Also, if you followed the Manual Installation process, then ensure your application has the related DependOn attribute with the hangfiremodule:

[DependsOn(typeof(AbpBackgroundJobsHangfireModule))]

0

priyankasynapxe created 14 days ago

I'm trying to implement hangfire but as soon as I put Queue attribute, my job is not executing,

pls help me to understand if any other step is required, I'm following https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire

Hi, did you configure a storage for Hangfire as follows in your module class?:
      context.Services.AddHangfire(config => 
      { 
          config.UseSqlServerStorage(configuration.GetConnectionString("Default")); 
      }); 
Also, if you followed the Manual Installation process, then ensure your application has the related DependOn attribute with the hangfiremodule:
[DependsOn(typeof(AbpBackgroundJobsHangfireModule))] 

Yes, I have added context.Services.AddHangfire(config => { config.UseSqlServerStorage(configuration.GetConnectionString("Default")); });

also, I used API CLI for installation abp add-package Volo.Abp.BackgroundJobs.HangFire

0

priyankasynapxe created 14 days ago

I'm trying to implement hangfire but as soon as I put Queue attribute, my job is not executing,

pls help me to understand if any other step is required, I'm following https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire

Hi, did you configure a storage for Hangfire as follows in your module class?:
      context.Services.AddHangfire(config =>  
      {  
          config.UseSqlServerStorage(configuration.GetConnectionString("Default"));  
      });  
Also, if you followed the Manual Installation process, then ensure your application has the related DependOn attribute with the hangfiremodule:
[DependsOn(typeof(AbpBackgroundJobsHangfireModule))]  
Yes, I have added
context.Services.AddHangfire(config => { config.UseSqlServerStorage(configuration.GetConnectionString("Default")); });

also, I used API CLI for installation abp add-package Volo.Abp.BackgroundJobs.HangFire

It is working when I'm using [Queue("default")], is it fine to use default queue ?

0

priyankasynapxe created 14 days ago

Can you please confirm one more thing, using hangfire will execute multiple jobs at the same time as multithreading?

0

EngincanV created 14 days ago

Support Team .NET Developer

Can you please confirm one more thing, using hangfire will execute multiple jobs at the same time as multithreading?

Yes, Hangfire can execute multiple jobs concurrently using background threads. It utilizes multithreading based on the number of worker threads configured in the Hangfire Server. By default, Hangfire can process several jobs in parallel depending on your server's capacity and configuration.

It is working when I'm using [Queue("default")], is it fine to use default queue ?

Yes, it's perfectly fine to use the "default" queue in Hangfire. In fact, if you don't specify a queue explicitly, Hangfire will automatically use the "default" queue for all background jobs.

But it should normally work with other queues, I'll investigate this.

0

priyankasynapxe created 10 days ago

Can you please confirm one more thing, using hangfire will execute multiple jobs at the same time as multithreading?

Yes, Hangfire can execute multiple jobs concurrently using background threads. It utilizes multithreading based on the number of worker threads configured in the Hangfire Server. By default, Hangfire can process several jobs in parallel depending on your server's capacity and configuration.

It is working when I'm using [Queue("default")], is it fine to use default queue ?

Yes, it's perfectly fine to use the "default" queue in Hangfire. In fact, if you don't specify a queue explicitly, Hangfire will automatically use the "default" queue for all background jobs.

But it should normally work with other queues, I'll investigate this.

Hi, can you confirm if I'll update hangfire code, it will be applicable for all background jobs added in application ?

Also, for dashbaord I want to add below code

private void ConfigureAuthentication(ServiceConfigurationContext context, IConfiguration configuration) { context.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme) .AddAbpJwtBearer(options => { options.Authority = configuration["AuthServer:Authority"]; options.RequireHttpsMetadata = configuration.GetValue<bool>("AuthServer:RequireHttpsMetadata"); options.Audience = "MyProjectName"; });

context.Services.AddAuthentication()
    .AddCookie("Cookies")
    .AddOpenIdConnect("oidc", options =>
    {
        options.Authority = configuration["AuthServer:Authority"];
        options.RequireHttpsMetadata = configuration.GetValue&lt;bool&gt;("AuthServer:RequireHttpsMetadata");
        options.ResponseType = OpenIdConnectResponseType.CodeIdToken;

        options.ClientId = configuration["AuthServer:ClientId"];
        options.ClientSecret = configuration["AuthServer:ClientSecret"];

        options.UsePkce = true;
        options.SaveTokens = true;
        options.GetClaimsFromUserInfoEndpoint = true;

        options.Scope.Add("roles");
        options.Scope.Add("email");
        options.Scope.Add("phone");
        options.Scope.Add("MyProjectName");
    });

}

can you please help me to understand what should be the ClientId and ClientSecret as in my existing setting I'm not using these 2 properties.

0

EngincanV created 10 days ago

Support Team .NET Developer

Hi, for dashboard authorization please refer to our documentation: https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire#dashboard-authorization

Hi, can you confirm if I'll update hangfire code, it will be applicable for all background jobs added in application ?

Yes. When you integrate Hangfire, it replaces the default background job manager, and Hangfire’s own job manager will be used instead.

0

priyankasynapxe created 10 days ago

Hi, for dashboard authorization please refer to our documentation: https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire#dashboard-authorization

Hi, can you confirm if I'll update hangfire code, it will be applicable for all background jobs added in application ?

Yes. When you integrate Hangfire, it replaces the default background job manager, and Hangfire’s own job manager will be used instead.

Hi, I referred document, I added permission, but it is not working, I'm getting 401 status code on accessing the dashboard even if permission is added, it is working in local but after deployment as my UI and backend is deployed on different server, I'm not able to access, what is the possible solution with this scenario?

0

EngincanV created 10 days ago

Support Team .NET Developer

Hi, for dashboard authorization please refer to our documentation: https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire#dashboard-authorization

Hi, can you confirm if I'll update hangfire code, it will be applicable for all background jobs added in application ?

Yes. When you integrate Hangfire, it replaces the default background job manager, and Hangfire’s own job manager will be used instead.

Hi, I referred document, I added permission, but it is not working, I'm getting 401 status code on accessing the dashboard even if permission is added, it is working in local but after deployment as my UI and backend is deployed on different server, I'm not able to access, what is the possible solution with this scenario?

Did you apply the all steps described at https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire#dashboard-authorization-in-api-projects?

1-)

2-)

If you applied, can you share the error log?

0

priyankasynapxe created 10 days ago

Hi, for dashboard authorization please refer to our documentation: https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire#dashboard-authorization

Hi, can you confirm if I'll update hangfire code, it will be applicable for all background jobs added in application ?

Yes. When you integrate Hangfire, it replaces the default background job manager, and Hangfire’s own job manager will be used instead.

Hi, I referred document, I added permission, but it is not working, I'm getting 401 status code on accessing the dashboard even if permission is added, it is working in local but after deployment as my UI and backend is deployed on different server, I'm not able to access, what is the possible solution with this scenario?

Did you apply the all steps described at https://abp.io/docs/latest/framework/infrastructure/background-jobs/hangfire#dashboard-authorization-in-api-projects?

1-)

2-)

If you applied, can you share the error log?

no, I'm not able to apply this as I'm not using ClientId and ClientSecret in my application anywhere, but it is expecting these 2 properties, can I skip it? or can you help me to understand what is the use of these properties?

0

EngincanV created 9 days ago

Support Team .NET Developer

Hi,

no, I'm not able to apply this as I'm not using ClientId and ClientSecret in my application anywhere

Are you sure about that? There should be clientId and clientSecret in your code, to define the OIDC client so you can login through your auth-server. Please search it through your application. (ClientId is typically probably is your application name, and clientSecret can be 1q2w3E* if you haven't changed it but, you should check your HttpApiHost project or DbMigrator project's appsettings.json file)

0

priyankasynapxe created 9 days ago

Hi,

no, I'm not able to apply this as I'm not using ClientId and ClientSecret in my application anywhere

Are you sure about that? There should be clientId and clientSecret in your code, to define the OIDC client so you can login through your auth-server. Please search it through your application. (ClientId is typically probably is your application name, and clientSecret can be 1q2w3E* if you haven't changed it but, you should check your HttpApiHost project or DbMigrator project's appsettings.json file)

we are using external login; can I use that ClientId and ClientSecret ?

0

EngincanV created 9 days ago

Support Team .NET Developer

Hi,

no, I'm not able to apply this as I'm not using ClientId and ClientSecret in my application anywhere

Are you sure about that? There should be clientId and clientSecret in your code, to define the OIDC client so you can login through your auth-server. Please search it through your application. (ClientId is typically probably is your application name, and clientSecret can be 1q2w3E* if you haven't changed it but, you should check your HttpApiHost project or DbMigrator project's appsettings.json file)

we are using external login; can I use that ClientId and ClientSecret ?

yes, if you are login through an external login provider, then you should pass that.

0

priyankasynapxe created 9 days ago

Hi,

no, I'm not able to apply this as I'm not using ClientId and ClientSecret in my application anywhere

Are you sure about that? There should be clientId and clientSecret in your code, to define the OIDC client so you can login through your auth-server. Please search it through your application. (ClientId is typically probably is your application name, and clientSecret can be 1q2w3E* if you haven't changed it but, you should check your HttpApiHost project or DbMigrator project's appsettings.json file)

we are using external login; can I use that ClientId and ClientSecret ?

yes, if you are login through an external login provider, then you should pass that.

I have updated my code with but I'm getting below error after deployment on sign in:

2025-04-16 17:42:47.809 +08:00 [INF] Error from RemoteAuthentication: Unable to unprotect the message.State.. 2025-04-16 17:42:47.810 +08:00 [ERR] An unhandled exception has occurred while executing the request. Microsoft.AspNetCore.Authentication.AuthenticationFailureException: An error was encountered while handling the remote login. ---> Microsoft.AspNetCore.Authentication.AuthenticationFailureException: Unable to unprotect the message.State. --- End of inner exception stack trace --- at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler1.HandleRequestAsync() at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at eFC.Web.eFCWebModule.<>c.<<OnApplicationInitialization>b__15_0>d.MoveNext() in C:\DIM\AprilRelease\eFC\src\src\eFC.Web\eFCWebModule.cs:line 471 --- End of stack trace from previous location --- at Volo.Abp.AspNetCore.Tracing.AbpCorrelationIdMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task) 2025-04-16 17:42:47.810 +08:00 [INF] CORS policy execution successful. 2025-04-16 17:42:47.813 +08:00 [INF] Executing endpoint 'Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared.Controllers.ErrorController.Index (Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared)' 2025-04-16 17:42:47.813 +08:00 [INF] Route matched with {action = "Index", controller = "Error", area = "", page = ""}. Executing controller action with signature System.Threading.Tasks.Task1[Microsoft.AspNetCore.Mvc.IActionResult] Index(Int32) on controller Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared.Controllers.ErrorController (Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared). 2025-04-16 17:42:47.816 +08:00 [INF] Executing ViewResult, running view ~/Views/Error/500.cshtml. 2025-04-16 17:42:47.827 +08:00 [INF] Executed ViewResult - view ~/Views/Error/500.cshtml executed in 11.268ms. 2025-04-16 17:42:47.827 +08:00 [INF] Executed action Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared.Controllers.ErrorController.Index (Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared) in 13.6018ms 2025-04-16 17:42:47.827 +08:00 [INF] Executed endpoint 'Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared.Controllers.ErrorController.Index (Volo.Abp.AspNetCore.Mvc.UI.Theme.Shared)' 2025-04-16 17:42:48.006 +08:00 [INF] Request finished HTTP/1.1 POST https://hisfincvsapp02/SIT/eFCApp/signin-oidc - 500 null text/html; charset=utf-8 199.4496ms 2025-04-16 17:42:48.036 +08:00 [INF] Request starting HTTP/1.1 GET https://hisfincvsapp02/SIT/eFCApp/__bundles/Lepton.Global.CB0D5B09BE6F106B497F8CE2353AE3CC.css?_v=638803933595759917 - null null 2025-04-16 17:42:48.036 +08:00 [INF] Request starting HTTP/1.1 GET https://hisfincvsapp02/SIT/eFCApp/__bundles/Lepton.Global.8C50097CE846D065D72E2542DAD1838D.js?_v=638803933601845153 - null null 2025-04-16 17:42:48.038 +08:00 [INF] Sending file. Request path: '/__bundles/Lepton.Global.CB0D5B09BE6F106B497F8CE2353AE3CC.css'. Physical path: 'N/A' 2025-04-16 17:42:48.039 +08:00 [INF] Sending file. Request path: '/__bundles/Lepton.Global.8C50097CE846D065D72E2542DAD1838D.js'. Physical path: 'N/A' 2025-04-16 17:42:48.045 +08:00 [INF] Request finished HTTP/1.1 GET

0

EngincanV created 9 days ago

Support Team .NET Developer

Hi, it's impossible for us to fix the problem with the shared logs. Actually, we are directly using Hangfire's own dashboard UI and you can check their documentation to understand the reason: https://docs.hangfire.io/en/latest/configuration/using-dashboard.html

Please refer to the related documentation, and if you still can't fix your problem, then we can assist you.

0

priyankasynapxe created 3 days ago

Hi, can you please confirm how can we manage servers on which hangfire will run, in my case, I have 2 web solution (added screenshot below for reference), 1 is for application and other is for interface, on interface solution background job is disabled and I have not added any hangfire code on WebModule.cs of eFC.Web_Int, I have added code only in WebModule.cs of eFC.Web. My app server and interface servers are different. eFC.Web is deployed on e.g. AppServer1, AppServer2 and eFC.Web_Int is deployed on e.g. InterfaceServer1, InterfaceServer2. My expectation is all the jobs should be processed by AppServer1, AppServer2 but seems like it is going all Interface servers too. How can I restrict that? also, whenever it is going on interface server it is giving below error, which is causing delay in job execution which is why I don't want job to execute in interface server.

0

enisn created 3 days ago

Support Team .NET Developer

Hi,

Can you share how you configured your Background Jobs in your each application? Including .Web and .Web_Int

Normally, if you disable job execution, it shouldn't even be triggered. It seems some misconfiguration.

0

priyankasynapxe created 2 days ago

Hi,

Can you share how you configured your Background Jobs in your each application? Including .Web and .Web_Int

Normally, if you disable job execution, it shouldn't even be triggered. It seems some misconfiguration.

I checked, this is happening as my application layer is shared between both solutions, I believe I need to split both solutions.

22 Answer(s)

ABP Live Training
Packages

Do you need assistance from an ABP expert?

Mastering
ABP Framework

How to run ABP background job parallelly #9106

22 Answer(s)

ABP Live TrainingPackages

Do you need assistance from an ABP expert?

MasteringABP Framework

ABP Live Training
Packages

Mastering
ABP Framework