Environment ABP: 10.0.1 (commercial + open-source packages) .NET: 10.0 Hosting: Kubernetes (staging / UAT) Distributed Event Bus: Volo.Abp.EventBus.Azure / Azure Service Bus Database: SQL Server (EF Core) Affected services: Synergy (ERP), WMS, Tile (same shared hosting module) Previous behavior: Same services on older ABP stack started in ~5–10 seconds
Summary After upgrading to ABP 10.0.1, on the first pod start after deploy the host hangs during await app.InitializeApplicationAsync(). The process stays Running but never reaches “listening”. After ~120 seconds we force-exit (our own startup deadline) so Kubernetes restarts the pod; the second start completes normally and the service becomes healthy.
This looks like a cold first-start / blocking init issue, not a permanent configuration failure.
Startup sequence (Synergy / ERP example)
SerilogConfigurationHelper.Configure(...); Log.Information("Starting ..."); var builder = WebApplication.CreateBuilder(args); builder.Host.AddAppSettingsSecretsJson().UseAutofac().UseSerilog(); await builder.AddApplicationAsync<SynergyServiceHttpApiHostModule>(); var app = builder.Build(); await app.InitializeApplicationAsync(); // <-- hangs here on first start await app.RunAsync(); // never reached on first start
Shared host module depends on:
NxPSharedHostingMicroservicesModule which depends on AbpEventBusAzureModule, Administration EF, SaaS EF, etc.
Azure Event Bus is configured via env (Azure__ServiceBus__Connections__Default__ConnectionString, Azure__EventBus__TopicName, Azure__EventBus__SubscriberName, Azure__EventBus__ConnectionName). Topic and subscriptions already exist.
Logs from first hang (Synergy)
[INF] Elasticsearch logging: enabled for https://.... Elasticsearch failures will not block startup.
[INF] NxP.SynergyService.HttpApi.Host startup deadline is 120 seconds...
[INF] Starting NxP.SynergyService.HttpApi.Host.
[INF] ... startup: configuring ABP modules (AddApplicationAsync).
[INF] ... startup: ABP modules configured. Building host.
[INF] ... startup: initializing application (middleware, Event Bus, DB check).
No further logs until our process exit after 120s.
We never reach:
Shared module log: dependency modules initialized. Waiting for the host to listen. starting hosted services (RunAsync)... Now listening on...
So the block is inside ABP module initialization (InitializeApplicationAsync), before Kestrel/RunAsync.
What we already ruled out
- Pending EF migrations – none; migration check is not the lasting blocker.
- Missing Service Bus topic/subscription – already provisioned.
- Elasticsearch – can hang if misconfigured; we made ES optional/non-blocking. Hang still occurs with ES correctly enabled.
- Redis distributed lock – Redis lock provider is not registered; ABP uses LocalAbpDistributedLock. Log text mentioning Redis was misleading on our side.
- Missing Auth metadata / Swagger OIDC – fixed where needed; not the current failure mode.
- Permanent bad config – second start after restart works without config change.
Pattern across services Tile, WMS, and Synergy (ERP) all show the same pattern on first deploy:
- Hang in InitializeApplicationAsync
- Forced exit ~120s
- Kubernetes restart
- Second start succeeds in a few seconds Suggests first cold initialization (likely Event Bus consumer connect / module init / first SQL under load), not a missing setting.
Questions for ABP team
- In ABP 10.0.1, is it expected that AbpEventBusAzureModule.OnApplicationInitialization / AzureDistributedEventBus.Initialize() can block application startup for a long time on first Service Bus connect, even when topic/subscription already exist?
- Is there a recommended way to defer Azure Event Bus consumer start until after the host is listening (or make init fail fast with timeout instead of hanging)?
- Are there known issues / fixes in 10.0.2 / 10.0.3 (or later) specifically for Event Bus / module init blocking InitializeApplicationAsync?
- Besides Event Bus, which built-in modules commonly perform long network/SQL work during OnApplicationInitialization that could block host start (settings static save, localization external store, background workers, etc.)?
- Recommended diagnostics: best way to log which module is currently initializing (we added an IModuleLifecycleContributor logger; is there a built-in ABP option)?
What we tried
- Synchronous console Serilog (avoid buffered logs on hang)
- Startup stage logs around AddApplicationAsync / InitializeApplicationAsync / RunAsync
- Service Bus client/admin retry timeouts reduced
- HostOptions.ServicesStartConcurrently = true (hang is before RunAsync, so this does not address current stage)
- Startup deadline exit so K8s can restart (workaround only)
- Per-module lifecycle logging (to identify exact module on next deploy)
Request Please advise:
- Whether this is a known regression/behavior in 10.0.1 Azure Event Bus / module startup
- Recommended configuration or code pattern so microservices become ready without depending on a kill/restart
- Whether upgrading to 10.0.3 (patch) or a later minor is expected to help this specific symptom
3 Answer(s)
-
0
InitializeApplicationAsync()runs ABP’s module lifecycle (OnPreApplicationInitialization/OnApplicationInitialization/OnPostApplicationInitialization) before Kestrel starts listening. So any module that does I/O (DB/network) during these phases can block the process from becoming “ready”.- In ABP microservice solutions, cold-start startup dependency failures / delays are considered expected in distributed systems, and the template explicitly relies on orchestrators (ABP Studio runner / Kubernetes) to restart services until dependencies are ready. This is documented as “Rapid Failures on System Startup” (even though your symptom is a hang, the underlying principle is the same: startup can be blocked by external dependencies).
- I could not verify (from available indexed sources/tools) a specific ABP 10.0.1 regression/bug in
Volo.Abp.EventBus.Azurethat would explain a ~120s hang only on first start, nor a documented/official way to “defer Azure Event Bus consumer start until after listening” for 10.0.x. So I can’t definitively confirm (1), (2), or (3) as written.
Practical guidance that is consistent with ABP’s documented microservice startup approach (and matches your observed “second start succeeds” pattern):
Treat the first-start dependency wait as normal and let Kubernetes handle it
- Increase
startupProbe/readinessProbe/livenessProbegrace periods so the pod isn’t killed by your own 120s deadline while ABP is still initializing. - Remove/relax the custom “startup deadline forced exit” if possible; otherwise make it long enough to cover worst-case cold-start dependency waits.
- Rationale: ABP’s microservice template expects services can fail/restart until their dependencies (DBs, other services) are available.
- Increase
Focus diagnostics on “which module is blocking initialization”
- Since the hang is inside
InitializeApplicationAsync(), instrumenting module lifecycle (as you started doing with anIModuleLifecycleContributorlogger) is the correct direction. - If the module lifecycle logger identifies the last module before the hang as the Azure Service Bus event bus module (or a module that starts background workers / inbox-outbox processors), you’ll have a concrete component to isolate further (e.g., temporarily remove that module dependency and confirm startup time returns to normal).
- Since the hang is inside
Most likely blocking categories during ABP initialization (generic, but common in microservices)
- Database connectivity / first query under load (connection pool warm-up, DNS, firewall, SQL availability).
- Distributed event bus connect/subscription initialization (network to Azure Service Bus, DNS, TLS, RBAC, throttling).
- Any startup logic you added in your own modules’
OnApplicationInitialization/OnPostApplicationInitialization(including “check migrations”, seed data, etc.).
Upgrade guidance
- I can’t claim a verified “fixed in 10.0.2/10.0.3” for Azure Event Bus startup blocking based on the currently retrievable sources/tools.
- Still, upgrading to the latest available patch in your major/minor line is generally advisable; however, I can’t provide a verified ABP source stating it resolves this exact symptom.
Sources (validated):
- https://abp.io/docs/10.2/solution-templates/microservice/database-configurations
- https://abp.io/docs/10.6/solution-templates/microservice/database-configurations
- https://abp.io/docs/10.7/solution-templates/microservice/database-configurations
This response was generated by AI. If it didn’t solve your problem, feel free to post a new answer — our support team will jump in to help you soon.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
I confirm that due to azure service bus, service hangs, I removed the azure service bus configuration and service started successfully, when I add azure service bus, it starts hang again.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) -
0
- Since removing Azure Service Bus configuration removes the hang, the blocker is inside ABP’s Azure distributed event bus initialization, which is executed during
app.InitializeApplicationAsync()(i.e., before Kestrel starts listening). - ABP does not document an official “start the Azure Service Bus consumer after the web host is listening” switch for 10.x, and I can’t verify a 10.0.1-specific regression/fix from available sources. What is verifiable is that ABP’s Azure integration is enabled by adding
Volo.Abp.EventBus.Azure/AbpEventBusAzureModule, and that microservice startup can be sensitive to infrastructure availability.
Solution
Ensure Kubernetes, not your app, is responsible for retrying cold starts
- Remove the “force-exit after 120s” workaround (or increase it substantially), and instead rely on Kubernetes
startupProbeto allow long cold starts. - If you must keep an upper bound, make it higher than worst-case Service Bus first-connection time (DNS/TLS/RBAC can be slow on first pod).
- Remove the “force-exit after 120s” workaround (or increase it substantially), and instead rely on Kubernetes
Fail-fast the Azure Service Bus connectivity at startup (so you get an error instead of a silent hang)
- Add an explicit “connectivity check” before
InitializeApplicationAsync()and log exceptions. Example pattern:- Create a
ServiceBusAdministrationClientand call a lightweight operation (e.g., get topic/subscription properties) with your own cancellation/timeout. - If it times out/fails, throw so the container restarts quickly and you get a visible failure reason in logs.
- Create a
- This doesn’t “defer consumer start”, but it converts “hang” into “fast crash with logs”, which is typically what you want in Kubernetes.
- Add an explicit “connectivity check” before
Add ABP module lifecycle logging around Azure event bus module
- Keep the
IModuleLifecycleContributorapproach you already started and log start/end timestamps for each module initialization step. This is the most direct way to prove exactly where it blocks (and whether it’s always the Azure event bus module).
- Keep the
Validate Azure Service Bus prerequisites that commonly cause long waits Even when entities exist, first-start hangs are often caused by infra-level issues that don’t immediately throw:
- DNS resolution delays inside the cluster (first-time egress / private DNS zone / CoreDNS under load).
- Network policy / firewall / outbound SNAT exhaustion.
- RBAC/managed identity delays if you’re using AAD (even if connection string-based, check KeyVault/secret injection timing).
- Service Bus throttling / transient failures (client retries can stretch startup).
Sources (validated)
- https://abp.io/docs/10.6/framework/infrastructure/event-bus/distributed/azure
- https://abp.io/docs/10.5/framework/infrastructure/event-bus/distributed/azure
- https://abp.io/docs/10.4/framework/infrastructure/event-bus/distributed/azure
- https://abp.io/docs/10.7/solution-templates/microservice/database-configurations
This response was generated by AI. If it didn’t solve your problem, feel free to post a new answer — our support team will jump in to help you soon.
Markdown supported.Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post) - Since removing Azure Service Bus configuration removes the hang, the blocker is inside ABP’s Azure distributed event bus initialization, which is executed during
