Ok thanks @liangshiwei
Ok. thanks. can you tell me if i have AsyncBackgroundWorkers, Is abp use HangfirePeriodicBackgroundWorkerAdapter to execute them? and which module is implementing BackgroundJobWorker? There are bunch of different nuget packages from Abp.
AbpBackgroundJobsEfCoreModule AbpBackgroundJobsDomainModule AbpBackgroundJobsDomainSharedModule AbpBackgroundJobsModule
only module i am sure to use is AbpBackgroundJobsAbstractions :)
Hello @liangshiwei, i figured it out. But i have some questions. The problem was: I had AbpBackgroundJobsEntityFrameworkCoreModule,AbpBackgroundJobsDomainModule and AbpBackgroundJobsDomainSharedModule in my app efcore, domain and domainshared module respectively. I have removed them instead i used
typeof(AbpBackgroundJobsModule),
typeof(AbpBackgroundJobsHangfireModule),
typeof(AbpBackgroundWorkersHangfireModule),
in my httpapi.host project. This fixed the problem however i didn't understand which module is implementing the DefaultBgJob implementation. Can you give me a hint about it probably it is efcoremodule since it should be the infra project. And why AbpBackgroundJobsHangfireModule didn't override it i didn't understand. But now it works in this order.
The second thing i want to ask is about HangfirePeriodicBackgroundWorkerAdapter<BackgroundJobWorker>.DoWorkAsync why is this work triggered every 5 secs? I believe that's the default value. How can that value be changed? Also is this worker are dispatching BackgroundJobs? What is the purpose of this worker can you elaborate? Is it possible to disable it and delete it?
Hello my question is about how to implement hangfire. I am using hangfire in my abp project. But i am having some weird behavior. To make the things simpler i can post some basic code for my module and hangfire implementation.
[DependsOn(
typeof(AbpAspNetCoreMvcUiMultiTenancyModule),
typeof(AbpAspNetCoreAuthenticationOpenIdConnectModule),
typeof(PopManagementModule),
typeof(AbpBlobStoringAzureModule),
typeof(AbpBackgroundJobsHangfireModule),
typeof(AbpBackgroundWorkersHangfireModule),
typeof(AbpAspNetCoreMvcUiLeptonXThemeModule),
typeof(DoohlinkSharedHostingMicroservicesModule)
)]
public class PopManagementHttpApiHostModule:AbpModule
{
//etc...
and here is how i configure hangfire.
private void ConfigureHangfire(ServiceConfigurationContext context, IConfiguration configuration)
{
context.Services.AddHangfire(config =>
{
var options = new PostgreSqlStorageOptions
{
DistributedLockTimeout = TimeSpan.FromMinutes(1)
};
config.UsePostgreSqlStorage(postgresSqlStorageConfig =>
{
postgresSqlStorageConfig.UseNpgsqlConnection(configuration.GetConnectionString("PopManagement"));
}, options);
});
}
and here is my project structure (part of my project)
so when i run the project everythings works fine. But the weird part is i don't see any background jobs enqueued in my hangfire dashboard with parameters.
what i mean by that is i have this background job.
public class PopJob : AsyncBackgroundJob<PopJobBto>, ITransientDependency
{
private readonly IPopRepository _popRepository;
private readonly IBlobContainer<PopCsvContainer> _popCsvContainer;
private readonly IGuidGenerator _guidGenerator;
public PopJob(IPopRepository popRepository, IBlobContainer<PopCsvContainer> popCsvContainer,
IGuidGenerator guidGenerator)
{
_popRepository = popRepository;
_popCsvContainer = popCsvContainer;
_guidGenerator = guidGenerator;
}
public override async Task ExecuteAsync(PopJobBto args)
{
var file = await _popCsvContainer.GetOrNullAsync(args.FileName);
if (file == null)
{
return;
}
var csvData = System.Text.Encoding.UTF8.GetString(GetBytesFromStream(file));
var items= ConvertFromCsv(csvData);
await _popRepository.InsertBulkAsync(items);
}
public sealed record PopJobBto(string FileName);
public class PopAppService : PopManagementAppService, IPopAppService
{
private readonly IBlobContainer<PopCsvContainer> _blobContainer;
private readonly IClock _clock;
private readonly IBackgroundJobManager _backgroundJobManager;
private readonly IDistributedCache<PopCacheItem, PopCacheItemKey> _distributedCache;
private readonly IPopRepository _popRepository;
private readonly IScreenRepository _screenRepository;
private readonly IFileRepository _fileRepository;
public PopAppService(IBlobContainer<PopCsvContainer> blobContainer, IClock clock,
IBackgroundJobManager backgroundJobManager, IDistributedCache<PopCacheItem, PopCacheItemKey> distributedCache,
IPopRepository popRepository, IScreenRepository screenRepository, IFileRepository fileRepository)
{
_blobContainer = blobContainer;
_clock = clock;
_backgroundJobManager = backgroundJobManager;
_distributedCache = distributedCache;
_popRepository = popRepository;
_screenRepository = screenRepository;
_fileRepository = fileRepository;
}
public async Task CreateAsync(CreatePopDto input)
{
//create csv file from the input and send it to azure blob storage
var csvData = ConvertToCsv(input.Items);
var csvBytes = GetCsvBytes(csvData);
// Generate the file name using the first item's DeviceId and the current date and time
if (!input.Items.Any())
{
return;
}
var deviceId = input.Items.First().DeviceId;
var dateTime = _clock.Now.ToString("yyyyMMdd_HHmmss");
var fileName = $"{deviceId}_{dateTime}.csv";
await _blobContainer.SaveAsync(fileName, csvBytes);
await _backgroundJobManager.EnqueueAsync(new PopJobBto(fileName));
}
so in hangfire i am expecting to see the file name and how it is serialized. But all i see is background workers.
I believe it still uses default fifo implementation cause i can see that it hits background job. But sometimes it hangs and never fire the background job, could be because of the distributed lock i suppose. What am i missing over here. Can you point me to the right direction?
ok thank you for explanation. I think that abp is easing so many things but at the same time it becomes very difficult for simple implementations. Maybe i should just create my own implementation for domain events instead of using local event bus. Anyway i will think about it. IDistributedEventBus is not the solution for me since i also use rabbitmq for my distributed messaging. I am closing this discussion over here. Thanks anyway.
Ok. So in my project i use LocalEventBus (for internal modules) and DistributedEventBus (for other api projects running with my main app). If i want to implement Inbox / Outbox pattern to my internal modules (with local events) i believe that is not possible with abp out of the box. Am i right about it? Inbox / Outbox pattern is only designed for distributed event bus?
If i didn't have any other api in my project i know that i could use DistributedEventBus. But that option no longer exists as i see it. I am aware with other complexities that it would bring if i deal with one dbcontext. But performance can be sth i can choose in that case. I believe i should implement that myself.
public virtual async Task CompleteAsync(CancellationToken cancellationToken = default)
{
if (_isRolledback)
{
return;
}
PreventMultipleComplete();
try
{
_isCompleting = true;
await SaveChangesAsync(cancellationToken);
DistributedEvents.AddRange(GetEventsRecords(DistributedEventWithPredicates));
LocalEvents.AddRange(GetEventsRecords(LocalEventWithPredicates));
while (LocalEvents.Any() || DistributedEvents.Any())
{
if (LocalEvents.Any())
{
var localEventsToBePublished = LocalEvents.OrderBy(e => e.EventOrder).ToArray();
LocalEventWithPredicates.Clear();
LocalEvents.Clear();
await UnitOfWorkEventPublisher.PublishLocalEventsAsync(
localEventsToBePublished
);
}
if (DistributedEvents.Any())
{
var distributedEventsToBePublished = DistributedEvents.OrderBy(e => e.EventOrder).ToArray();
DistributedEventWithPredicates.Clear();
DistributedEvents.Clear();
await UnitOfWorkEventPublisher.PublishDistributedEventsAsync(
distributedEventsToBePublished
);
}
await SaveChangesAsync(cancellationToken);
LocalEvents.AddRange(GetEventsRecords(LocalEventWithPredicates));
DistributedEvents.AddRange(GetEventsRecords(DistributedEventWithPredicates));
}
await CommitTransactionsAsync(cancellationToken);
IsCompleted = true;
await OnCompletedAsync();
}
catch (Exception ex)
{
_exception = ex;
throw;
}
}
so i should implement my own logic for local events here instead so i shouldn't publish the events maybe bg job can publish it after the transaction complete accordingly? That is going to be difficult for each module since each module needs to have the same code duplication for inbox / outbox pattern.
ok i got it Enis. Thank you for the explanation. So when you call unitOfWork.complete() it closes all the dbcontexts inside the unitofwork so before it comes to there if there is an exception all the dbcontexts are gonna be rolled back.
Just one more question. Does it have any performance problem since it is dealing with multiple db contexts with separate connection strings in one request? Cause i have seen in so many examples, they do not deal with unitofwork in multiple databases instead they have implemented inbox/outbox pattern and closing the transactions for each modules. So you can think they use separate unit of work for each module and use inbox/outbox pattern.
Hello Enis, Thanks for the reply. I understand the concept. I just want to give one example since i don't know how abp would react to it. Let's say I have one separate database for each module. Let's assume CreateOrder() appservice have been called from Ordering Module and as a side effect of the module i raised a local event OrderCreated. On Ticketing Module i want to issue a ticket for that event. OrderCreatedEventHandler catched the event and trying to insert a new ticket to Ticketing Database(which is separate from Ordering Database). If you can not reach the Ticketing Database at that time what is happening? Is UnitOfWork rolling back so the Ordering insert is also rolled back or eventual consistency is broken. I believe it is gonna be broken as you mentioned you can not create transaction between separate databases. Just correct me if i am wrong.
And if it is broken. What can i do to solve the problem? My solution could be using inbox / outbox pattern in that case. Even if it is a local event if you can implement inbox / outbox pattern, then you can raise the same event from inbox messages again after some time and correct the state for Ticketing module. Is Abp supports that case? I know Abp supports it for DistributedEventBus. Can i use it also for LocalEventBus? And is there any sample for it?
PS:
If you use LocalEventBus, it will be a local transaction, and whenever an exception occurs in the consumer side, the transaction from the publisher side will be rolled back.
so you say even if it is a different database, unitofwork is going to rollback the transaction? are you sure about it? Isn't unitofwork calling savechanges() for each dbcontext and closing the transaction at the end of unitofwork?
Hello i have a very general question about how the unitofwork is working in Abp. If i use multiple db context with separate connection strings in my modular monolith app and if i want to do multiple operations with one unitofwork. Is it going to be a problem? since each of the modules are going to have different databases, i want to ask how does transaction is going to be handled. If i raise a domain event and as a side effect in the same scope i want to also insert a record on another database server? Or if the app has a separate databases for each module on one database server. Thank you for the assistance.
Hello I figured it out @yekalkan. The problem was, I was debugging it as https instead of http. That’s why it didn’t hit the breakpoint. But it could be nice for abp studio to get the values from the pod instead of yaml file