Starts in:
2 DAYS
14 HRS
25 MIN
15 SEC
Starts in:
2 D
14 H
25 M
15 S
Open Closed

System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached. #8226


User avatar
0
s.beckers created
  • ABP Framework version: v8.1.4
  • UI Type: Angular
  • Database System: EF Core (SQL Server)
  • Tiered (for MVC) or Auth Server Separated (for Angular): no
  • Exception message and full stack trace:

The following exception occurs when our API is running for a few days in production. Logged out users are not able to login anymore due to this exception which results in a 500 error page instead of the login page.

We're hosting the API as a published release build in a linux docker container. It seems that the BundleManager still watches for file changes? Is this correct behaviour in a release build? Restarting the API container only fixes the issue temporary.

2024-11-05 09:21:35Z [ERR] Connection id ""0HN7TAA59SCFI"", Request id ""0HN7TAA59SCFI:00000003"": An unhandled exception was thrown by the application.
System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached.
   at System.IO.FileSystemWatcher.StartRaisingEvents()
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.TryEnableFileSystemWatcher()
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.CreateFileChangeToken(String filter)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider.Watch(String filter)
   at Microsoft.Extensions.FileProviders.CompositeFileProvider.Watch(String pattern)
   at Volo.Abp.AspNetCore.VirtualFileSystem.WebContentFileProvider.Watch(String filter)
   at Microsoft.Extensions.FileProviders.CompositeFileProvider.Watch(String pattern)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.BundleManager.WatchChanges(BundleCacheItem cacheValue, List`1 files, String bundleRelativePath)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.BundleManager.<>c__DisplayClass17_0.<AddToBundleCache>b__0()
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.BundleCache.GetOrAdd(String bundleName, Func`1 factory)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.BundleManager.AddToBundleCache(String bundleName, IBundler bundler, List`1 bundleFiles)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.BundleManager.GetBundleFilesAsync(BundleConfigurationCollection bundles, String bundleName, IBundler bundler)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.BundleManager.GetStyleBundleFilesAsync(String bundleName)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.TagHelpers.AbpTagHelperStyleService.GetBundleFilesAsync(String bundleName)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.TagHelpers.AbpTagHelperResourceService.ProcessAsync(ViewContext viewContext, TagHelper tagHelper, TagHelperContext context, TagHelperOutput output, List`1 bundleItems, String bundleName)
   at Volo.Abp.AspNetCore.Mvc.UI.Bundling.TagHelpers.AbpBundleTagHelperService`2.ProcessAsync(TagHelperContext context, TagHelperOutput output)
   at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.<RunAsync>g__Awaited|0_0(Task task, TagHelperExecutionContext executionContext, Int32 i, Int32 count)
   at AspNetCoreGeneratedDocument.Themes_Default_Layouts_Account.<>c__DisplayClass18_0.<<ExecuteAsync>b__0>d.MoveNext() in /src/modules/Fuzed.DefaultTheme/src/Fuzed.AspNetCore.Mvc.UI.Theme.Default/Themes/Default/Layouts/Account.cshtml:line 41
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperExecutionContext.SetOutputContentAsync()
   at AspNetCoreGeneratedDocument.Themes_Default_Layouts_Account.ExecuteAsync() in /src/modules/Fuzed.DefaultTheme/src/Fuzed.AspNetCore.Mvc.UI.Theme.Default/Themes/Default/Layouts/Account.cshtml:line 26
   at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context)
   at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts)
   at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderLayoutAsync(ViewContext context, ViewBufferTextWriter bodyWriter)
   at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable`1 statusCode)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable`1 statusCode)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeResultFilters>g__Awaited|28_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Volo.Abp.AspNetCore.Serilog.AbpSerilogMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Volo.Abp.AspNetCore.Auditing.AbpAuditingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Volo.Abp.AspNetCore.Auditing.AbpAuditingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Volo.Abp.AspNetCore.Security.Claims.AbpDynamicClaimsMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Volo.Abp.AspNetCore.Uow.AbpUnitOfWorkMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Volo.Abp.AspNetCore.ExceptionHandling.AbpExceptionHandlingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Volo.Abp.AspNetCore.ExceptionHandling.AbpExceptionHandlingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Volo.Abp.AspNetCore.MultiTenancy.MultiTenancyMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Builder.ApplicationBuilderAbpOpenIddictMiddlewareExtension.<>c__DisplayClass0_0.<<UseAbpOpenIddictValidation>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Volo.Abp.AspNetCore.Security.AbpSecurityHeadersMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Fuzed.FuzedHttpApiHostModule.<>c.<<OnApplicationInitialization>b__1_0>d.MoveNext() in /src/src/Fuzed.HttpApi.Host/FuzedHttpApiHostModule.cs:line 157
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.RequestLocalization.AbpRequestLocalizationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

5 Answer(s)
  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    You can try

    1. https://stackoverflow.com/a/60727988/6680813
    2. Override the BundleManager to remove the WatchChanges

    Thanks.

  • User Avatar
    0
    s.beckers created

    While investigating the bundling I saw a lot of logs related to the bundling. Can you check if this is correct, since I see a lot of the same files getting bundled and minified? Or are these from multiple requests? Also, notice the timestamps.

    https://cdn.fuzed.app/share/abp/main-http-api-6f5c75958b.log

    Thanks!

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    hi

    The logs related to WatchChanges

    You can Override the BundleManager to remove the WatchChanges and check the logs again.

  • User Avatar
    0
    s.beckers created

    Hi

    I'll try that.

    Thanks!

  • User Avatar
    0
    maliming created
    Support Team Fullstack Developer

    ok

Made with ❤️ on ABP v9.1.0-preview. Updated on November 20, 2024, 13:06