Configure Hangfire with In-Memory Provider for Integration Testing #8388 | Support

berly created 6 days ago

I am using Hangfire for background jobs and workers in my application. The current configuration is set up in my ApplicationModule as follows:

public override async Task OnApplicationInitializationAsync(
    ApplicationInitializationContext context)
{
    await context.AddBackgroundWorkerAsync<EmailScannerWorker>();
}

public override void ConfigureServices(ServiceConfigurationContext context)
{
    Configure<AbpAutoMapperOptions>(options => { options.AddMaps<AccuseReceptionCommandeApplicationModule>(); });
    ConfigureHangfire(context);
}

private void ConfigureHangfire(ServiceConfigurationContext context)
{
    context.Services.AddHangfire(config =>
    {
        config.UsePostgreSqlStorage(options =>
        {
            options.UseNpgsqlConnection(context.Services.GetConfiguration().GetConnectionString("Default"));
        });
    });

    Configure<AbpHangfireOptions>(options =>
    {
        options.ServerOptions = new BackgroundJobServerOptions() { WorkerCount = 3 };
    });
}

I want to configure Hangfire to use an in-memory provider for integration tests, similar to how EF Core uses SQLite in-memory databases for testing. This would allow me to reproduce the behavior of the production code while running tests without relying on a PostgreSQL database.

Could you provide guidance or examples on how to set up Hangfire for this use case? Thank you!

13 Answer(s)

0

liangshiwei created 6 days ago
Support Team Fullstack Developer

Hi,

It's easy to do, you can install Hangfire.InMemory package and configure it. https://github.com/HangfireIO/Hangfire.InMemory
0

berly created 5 days ago

Hi,

Thank you for your suggestion regarding the Hangfire.InMemory package. I have already successfully configured Hangfire to use the In-Memory provider for testing purposes.

The challenge I am facing is not about setting up the In-Memory provider itself but about configuring my application to use PostgreSQL in production while switching to the In-Memory provider specifically for integration tests.

I am looking for a solution that allows dynamic configuration, similar to how EF Core supports SQLite in-memory databases for integration testing while using a different database provider (e.g., PostgreSQL) for production. Ideally, I want the integration tests to override the default configuration without modifying the core application setup.

Could you provide guidance on achieving this type of setup for Hangfire?

Thank you for your help!
0

liangshiwei created 5 days ago
Support Team Fullstack Developer

Sorry I don't get it.

for test, you can configure it in the ..TestBaseModule

berly created 5 days ago

Thank you for your reply.

The issue I am facing is that when running the tests, the ConfigureServices method in my ApplicationModule is executed, and it attempts to configure Hangfire using PostgreSQL with the line:

options.UseNpgsqlConnection(context.Services.GetConfiguration().GetConnectionString("Default")); This fails during the tests, as expected, because I should not be using PostgreSQL in the test environment but rather the In-Memory provider.

What I need is a way to override the Hangfire configuration in the test environment to use the In-Memory provider, while still keeping the PostgreSQL configuration for the application in production.

public class AccuseReceptionCommandeApplicationModule : AbpModule
{
    public override async Task OnApplicationInitializationAsync(
        ApplicationInitializationContext context)
    {
        await context.AddBackgroundWorkerAsync<EmailScannerWorker>();
    }

    public override void ConfigureServices(ServiceConfigurationContext context)
    {
        Configure<AbpAutoMapperOptions>(options => { options.AddMaps<AccuseReceptionCommandeApplicationModule>(); });
        ConfigureHangfire(context);
    }

    private void ConfigureHangfire(ServiceConfigurationContext context)
    {
        context.Services.AddHangfire(config =>
        {
            config.UsePostgreSqlStorage(options =>
            {
                options.UseNpgsqlConnection(context.Services.GetConfiguration().GetConnectionString("Default"));
            });
        });

        Configure<AbpHangfireOptions>(options =>
        {
            options.ServerOptions = new BackgroundJobServerOptions() { WorkerCount = 3 };
        });
    }
}

Should I add the following in the TestBaseModule to override the Hangfire configuration for tests?

context.Services.AddHangfire(config => config.UseInMemoryStorage());

If so, how can I ensure this configuration takes precedence over the PostgreSQL configuration in the ApplicationModule when running tests?

Thank you again for your guidance!

liangshiwei created 5 days ago

Support Team Fullstack Developer

Hi,

You can consider using the appsettings

{
    "Hangfire": {
        "Provider" : "in-memory"
    }
}

{
    "Hangfire": {
        "Provider" : "postgreSql"
    }
}

private void ConfigureHangfire(ServiceConfigurationContext context)
{
    var configuration = context.Services.GetConfiguration();
    context.Services.AddHangfire(config =>
    {
        if(configuration["Hangfire:Provider"] == "in-memory")
        {
            ....
        }
        ....
    });

    Configure(options =>
    {
        options.ServerOptions = new BackgroundJobServerOptions() { WorkerCount = 3 };
    });
}

berly created 5 days ago

Thank you for the solution; it's a great idea, and it works for dynamically selecting the Hangfire provider based on the configuration.

However, I am now facing a new issue when running multiple integration tests in sequence. If I run the tests one by one, they pass without any problems. But when I run multiple tests (more than two) together, the first test succeeds, and the subsequent ones fail with the following error:

An error occurred during the initialize Volo.Abp.Modularity.OnApplicationInitializationModuleLifecycleContributor phase of the module Volo.Abp.Identity.AbpIdentityProDomainModule, Volo.Abp.Identity.Pro.Domain, Version=9.0.0.0, Culture=neutral, PublicKeyToken=null: Cannot access a disposed object.

It seems like something is being disposed of between tests, which causes the failure. Could this be related to the Hangfire configuration, or is there something else I need to adjust in my test setup to handle multiple tests properly?

Volo.Abp.AbpInitializationException: An error occurred during the initialize Volo.Abp.Modularity.OnApplicationInitializationModuleLifecycleContributor...

Volo.Abp.AbpInitializationException
An error occurred during the initialize Volo.Abp.Modularity.OnApplicationInitializationModuleLifecycleContributor phase of the module Volo.Abp.Identity.AbpIdentityProDomainModule, Volo.Abp.Identity.Pro.Domain, Version=9.0.0.0, Culture=neutral, PublicKeyToken=null: Cannot access a disposed object.
Object name: 'Hangfire.InMemory.State.Dispatcher`1[[System.UInt64, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]'.. See the inner exception for details.
   at Volo.Abp.Modularity.ModuleManager.InitializeModules(ApplicationInitializationContext context)
   at Volo.Abp.AbpApplicationBase.InitializeModules()
   at Volo.Abp.AbpApplicationWithExternalServiceProvider.Initialize(IServiceProvider serviceProvider)
   at Volo.Abp.Testing.AbpIntegratedTest`1..ctor()
   at Manuloc.AccuseReceptionCommande.AccuseReceptionCommandeTestBase`1..ctor()
   at Manuloc.AccuseReceptionCommande.AccuseReceptionCommandeApplicationTestBase`1..ctor()
   at Manuloc.AccuseReceptionCommande.Samples.SampleAppServiceTests`1..ctor() in C:\Users\vivie\RiderProjects\ARC\test\Manuloc.AccuseReceptionCommande.Application.Tests\Samples\SampleAppServiceTests.cs:line 19
   at Manuloc.AccuseReceptionCommande.EntityFrameworkCore.Applications.EfCoreSampleAppServiceTests..ctor()
   at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean wrapExceptions)

System.ObjectDisposedException
Cannot access a disposed object.
Object name: 'Hangfire.InMemory.State.Dispatcher`1[[System.UInt64, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]'.
   at Hangfire.InMemory.State.Dispatcher`1.ThrowObjectDisposedException() in /_/src/Hangfire.InMemory/State/Dispatcher.cs:line 158
   at Hangfire.InMemory.State.Dispatcher`1.QueryReadAndWait[TCommand,T](TCommand query, Func`3 func) in /_/src/Hangfire.InMemory/State/Dispatcher.cs:line 95
   at Hangfire.InMemory.InMemoryConnection`1.GetAllEntriesFromHash(String key) in /_/src/Hangfire.InMemory/InMemoryConnection.cs:line 408
   at Hangfire.RecurringJobExtensions.GetOrCreateRecurringJob(IStorageConnection connection, String recurringJobId) in C:\projects\hangfire-525\src\Hangfire.Core\RecurringJobExtensions.cs:line 62
   at Hangfire.RecurringJobManager.AddOrUpdate(String recurringJobId, Job job, String cronExpression, RecurringJobOptions options) in C:\projects\hangfire-525\src\Hangfire.Core\RecurringJobManager.cs:line 115
   at Hangfire.RecurringJob.AddOrUpdate(String recurringJobId, String queue, Expression`1 methodCall, String cronExpression, RecurringJobOptions options) in C:\projects\hangfire-525\src\Hangfire.Core\RecurringJob.cs:line 568
   at Volo.Abp.BackgroundWorkers.Hangfire.HangfireBackgroundWorkerManager.AddAsync(IBackgroundWorker worker, CancellationToken cancellationToken)
   at Volo.Abp.Identity.AbpIdentityProDomainModule.OnApplicationInitializationAsync(ApplicationInitializationContext context)
   at Nito.AsyncEx.Synchronous.TaskExtensions.WaitAndUnwrapException(Task task)
   at Nito.AsyncEx.AsyncContext.&lt;&gt;c__DisplayClass15_0.&lt;Run&gt;b__0(Task t)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Nito.AsyncEx.Synchronous.TaskExtensions.WaitAndUnwrapException(Task task)
   at Nito.AsyncEx.AsyncContext.Run(Func`1 action)
   at Volo.Abp.Threading.AsyncHelper.RunSync(Func`1 action)
   at Volo.Abp.Identity.AbpIdentityProDomainModule.OnApplicationInitialization(ApplicationInitializationContext context)
   at Volo.Abp.Modularity.OnApplicationInitializationModuleLifecycleContributor.Initialize(ApplicationInitializationContext context, IAbpModule module)
   at Volo.Abp.Modularity.ModuleManager.InitializeModules(ApplicationInitializationContext context)

0

liangshiwei created 3 days ago
Support Team Fullstack Developer

Hi,

I can't reproduce the problem
0

berly created 2 days ago

Hi,

Thank you for your response. Since you are unable to reproduce the issue, would it be possible to share your email so I can invite you to the project? This way, you can pull the code and run the tests directly to observe the problem.

Thank you for your help!
0

liangshiwei created 2 days ago
Support Team Fullstack Developer

Hi,

okay, my email is shiwei.liang@volosoft.com
0

berly created yesterday

Hi Shiwei,

Thank you for sharing your email. I just added you to my directory. Thanks again!
0

liangshiwei created yesterday
Support Team Fullstack Developer

Hi,

Could you use https://wetransfer.com/
0

berly created yesterday

We can't use WeTransfer sorry.

I send you an email with the link to download.
0

liangshiwei created 19 hours ago
Support Team Fullstack Developer

Hi,

Seems this is a problem with Hangfire https://github.com/HangfireIO/Hangfire/issues/2437

ABP can't do nothing here.

Have an answer to this question? Log in and write your answer.