Hello,
We are using DistributedEventBus with Confluent Kafka.
We have wrapped _distributedEventBus.PublishAsync(event) in a try-catch block, but it doesn't catch the exception when it can't connect to Confluence Kafka. (Following your suggestion, use try-catch in https://support.abp.io/QA/Questions/3079/IDistributedEventBus-throw-exception-when-it-can-not-connect-to-Kafka)
try
{
await _distributedEventBus.PublishAsync(eventData);
}
catch (Exception ex)
{
_logger.LogError(ex.Message);
throw;
}
Additionally, we need to wait for 5 more minutes, then application throw exception as "An internal error occurred during your request!". For the request, it took too much time.
Q1: Why can't we catch the exception? Q2: How to avoid waiting for too long?
- ABP Framework version: v7.1.1
- UI Type: Flutter
- Database System: EF Core (SQL Server) / MongoDB
- Tiered (for MVC) or Auth Server Separated (for Angular):
- Exception message and full stack trace:
- Steps to reproduce the issue:
16 Answer(s)
-
0
Hi,
First:
The event is published in a transaction, it will not be published immediately, but after the transaction is completed, if you want to catch exceptions, you can try it.
try { await _distributedEventBus.PublishAsync(new MyEventData() { Name = "test" }, onUnitOfWorkComplete:false); //Or //await UnitOfWorkManager.Current.CompleteAsync(); } catch(Exception e) { Logger.LogInformation("PublishAsync error...............: " + e.StackTrace); }Second:
By default, the message time out is
300000ms: https://docs.confluent.io/platform/current/clients/confluent-kafka-dotnet/_site/api/Confluent.Kafka.ProducerConfig.html#Confluent_Kafka_ProducerConfig_MessageTimeoutMsThis means that it will wait for the end of the timeout to throw an exception.
You can config the timeout:
Configure<AbpKafkaOptions>(options => { options.ConfigureProducer = config => { config.MessageTimeoutMs = 60; }; });Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
Thanks, let me try it.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
ok
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
Test OK.
There is another question： Can I configure ProducerConfig and ConsumerConfig using the appsettings.json file? Or can they only be configured using the Options Classes? https://docs.abp.io/en/abp/latest/Distributed-Event-Bus-Kafka-Integration
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
Hi,
Or can they only be configured using the Options Classes?
Yes
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
ok, thanks
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
:)
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
After our discussion, modifying the MessageTimeoutMs is not the best option for us because it's difficult to set an appropriate timeout.
We would like to handle the issue of not being able to connect to the Kafka server and do not want the API to block. Currently, we have configured the settings as follows:
Configure<AbpKafkaOptions>(options => { options.ConfigureProducer = config => { config.EnableIdempotence = true; config.MessageSendMaxRetries = 3; config.RetryBackoffMs = 1000; }; });After the retries are exhausted, we still need to wait for the MessageTimeoutMs time.(default = 300000ms) Is it possible to prioritize the retry time, so that an error is reported when the retries are exhausted?
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
Hi,
This has nothing to do with ABP. :). This is the behavior of Kafka API
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
Got it, thanks.
In the event of connection issues with the Kafka server, what would you recommend doing?
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
Hi,
You should make sure the Kafka server is highly available.
When there is a problem with the connection it means there are serious problems with your application or Kafka server, and you must fix them. there is no other way.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
The strategies mentioned above are things we will do, but I would like to know if ABP has any mechanisms to handle sudden Kafka server connection failures, so that the API can operate smoothly without blocking.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
Hi,
No, you must do it yourself.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
You can consider using the Outbox/Inbox pattern, It can ensure that event data will not be lost when the Kafka server is in error.
https://docs.abp.io/en/abp/latest/Distributed-Event-Bus#outbox-inbox-for-transactional-events
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
OK, thank you.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
:)
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)
