Micronaut's Netty server: OOM error : Leak suspect in io.netty.buffer.PoolThreadCache and io.netty.channel.nio.NioEventLoop

huangapple
未分类评论52阅读模式
英文:

Micronaut's Netty server: OOM error : Leak suspect in io.netty.buffer.PoolThreadCache and io.netty.channel.nio.NioEventLoop

问题

有人能否请发表意见，是否在使用Micronaut的Netty服务器时遇到了OOM错误？

Eclipse内存分析器显示的泄漏嫌疑点指向了io.netty.buffer.PoolThreadCache和io.netty.channel.nio.NioEventLoop。

  1. 由“jdk.internal.loader.ClassLoaders$AppClassLoader @ 0x71b769e00”加载的“io.netty.buffer.PoolThreadCache”的232个实例占用45,436,224（46.54%）字节。
  3. 由“jdk.internal.loader.ClassLoaders$AppClassLoader @ 0x71b769e00”加载的“io.netty.channel.nio.NioEventLoop”的233个实例占用25,004,368（25.61%）字节。

堆栈跟踪：

  1. [7324.589s][warning][os,thread] 启动线程失败 - pthread_create失败（EAGAIN），属性为：stacksize: 1024k，guardsize: 0k，detached。
  2. [03:32:21.838] ERROR i.m.s.DefaultTaskExceptionHandler - 调用bean [io.micronaut.management.health.monitor.HealthMonitorTask@4e56d6cb]的计划任务时出错，无法创建本机线程：可能是内存不足或进程/资源限制已达到
  3. java.lang.OutOfMemoryError：无法创建本机线程：可能是内存不足或进程/资源限制已达到
  4.     at java.base/java.lang.Thread.start0(Native Method)
  5.     at java.base/java.lang.Thread.start(Thread.java:803)
  6.     at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:937)
  7.     at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1354)
  8.     at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
  9.     at io.micronaut.scheduling.instrument.InstrumentedExecutorService.submit(InstrumentedExecutorService.java:89)
  10.     at io.micronaut.core.async.publisher.AsyncSingleResultPublisher$ExecutorServiceSubscription.request(AsyncSingleResultPublisher.java:98)
  11.     at io.reactivex.internal.operators.flowable.FlowableFlatMap$InnerSubscriber.onSubscribe(FlowableFlatMap.java:656)
  12.     at io.micronaut.core.async.publisher.AsyncSingleResultPublisher.subscribe(AsyncSingleResultPublisher.java:59)
  13.     at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onNext(FlowableFlatMap.java:163)
  14.     at io.reactivex.internal.operators.flowable.FlowableFromIterable$IteratorSubscription.slowPath(FlowableFromIterable.java:236)
  15.     at io.reactivex.internal.operators.flowable.FlowableFromIterable$BaseRangeSubscription.request(FlowableFromIterable.java:124)
  16.     at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onSubscribe(FlowableFlatMap.java:117)
  17.     at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribe(FlowableFromIterable.java:69)
  18.     at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribeActual(FlowableFromIterable.java:47)
  19.     at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  20.     at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:53)
  21.     at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  22.     at io.reactivex.internal.operators.flowable.FlowableFilter.subscribeActual(FlowableFilter.java:37)
  23.     at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  24.     at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
  25.     at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  26.     at io.micronaut.management.health.monitor.HealthMonitorTask.monitor(HealthMonitorTask.java:98)
  27.     at io.micronaut.management.health.monitor.$HealthMonitorTaskDefinition$$exec1.invokeInternal(Unknown Source)
  28.     at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
  29.     at io.micronaut.inject.DelegatingExecutableMethod.invoke(DelegatingExecutableMethod.java:76)
  30.     at io.micronaut.scheduling.processor.ScheduledMethodProcessor.lambda$process$5(ScheduledMethodProcessor.java:125)
  31.     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
  32.     at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
  33.     at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
  34.     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  35.     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  36.     at java.base/java.lang.Thread.run(Thread.java:834)
  37. [7624.677s][warning][os,thread] 启动线程失败 - pthread_create失败（EAGAIN），属性为：stacksize: 1024k，guardsize: 0k，detached。
  38. [03:37:21.925] WARN  i.m.caffeine.cache.BoundedLocalCache - 提交维护任务时抛出异常
  39. java.lang.OutOfMemoryError: 无法创建本机线程：可能是内存不足或进程/资源限制已达到
  40.     at java.base/java.lang.Thread.start0(Native Method)
  41.     at java.base/java.lang.Thread.start(Thread.java:803)
  42.     at java.base/java.util.concurrent.ForkJoinPool.createWorker(ForkJoinPool.java:1329)
  43.     at java.base/java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1352)
  44.     at java.base/java.util.concurrent.ForkJoinPool.signalWork(ForkJoinPool.java:1476)
  45.     at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.push(ForkJoinPool.java:852)
  46.     at java.base/java.util.concurrent.ForkJoinPool.externalSubmit(ForkJoinPool.java:1919)
  47.     at java.base/java.util.concurrent.ForkJoinPool.execute(ForkJoinPool.java:2453)
  48.     at io.micronaut.caffeine.cache.BoundedLocalCache.scheduleDrainBuffers(BoundedLocalCache.java:1419)
  49.     at io.micronaut.caffeine.cache.BoundedLocalCache.performCleanUp(BoundedLocalCache.java:1452)
  50.     at io.micronaut.caffeine.cache.BoundedLocalCache$PerformCleanupTask.run(BoundedLocalCache.java:3328)
  51.     at io.micronaut.caffeine.cache.BoundedLocalCache$PerformCleanupTask.exec(BoundedLocalCache.java:3315)
  52.     at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
  53.     at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
  54.     at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
  55.     at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594
  57. <details>
  58. <summary>英文:</summary>
  60. Can anyone please chime in whether this issue was encountered OOM error while using Micronaut&#39;s Netty server? 
  62. The leak suspects shown in Eclipse memory analyzer points to 
  63. `io.netty.buffer.PoolThreadCache` and `io.netty.channel.nio.NioEventLoop`

232 instances of "io.netty.buffer.PoolThreadCache", loaded by "jdk.internal.loader.ClassLoaders$AppClassLoader @ 0x71b769e00" occupy 45,436,224 (46.54%) bytes.

233 instances of "io.netty.channel.nio.NioEventLoop", loaded by "jdk.internal.loader.ClassLoaders$AppClassLoader @ 0x71b769e00" occupy 25,004,368 (25.61%) bytes.

  2. Stacktrace:
  3. ------------

[7324.589s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
[03:32:21.838] ERROR i.m.s.DefaultTaskExceptionHandler - Error invoking scheduled task for bean [io.micronaut.management.health.monitor.HealthMonitorTask@4e56d6cb] unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:937)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1354)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
at io.micronaut.scheduling.instrument.InstrumentedExecutorService.submit(InstrumentedExecutorService.java:89)
at io.micronaut.core.async.publisher.AsyncSingleResultPublisher$ExecutorServiceSubscription.request(AsyncSingleResultPublisher.java:98)
at io.reactivex.internal.operators.flowable.FlowableFlatMap$InnerSubscriber.onSubscribe(FlowableFlatMap.java:656)
at io.micronaut.core.async.publisher.AsyncSingleResultPublisher.subscribe(AsyncSingleResultPublisher.java:59)
at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onNext(FlowableFlatMap.java:163)
at io.reactivex.internal.operators.flowable.FlowableFromIterable$IteratorSubscription.slowPath(FlowableFromIterable.java:236)
at io.reactivex.internal.operators.flowable.FlowableFromIterable$BaseRangeSubscription.request(FlowableFromIterable.java:124)
at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onSubscribe(FlowableFlatMap.java:117)
at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribe(FlowableFromIterable.java:69)
at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribeActual(FlowableFromIterable.java:47)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:53)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableFilter.subscribeActual(FlowableFilter.java:37)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
at io.reactivex.Maybe.subscribe(Maybe.java:4290)
at io.micronaut.management.health.monitor.HealthMonitorTask.monitor(HealthMonitorTask.java:98)
at io.micronaut.management.health.monitor.$HealthMonitorTaskDefinition$$exec1.invokeInternal(Unknown Source)
at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
at io.micronaut.inject.DelegatingExecutableMethod.invoke(DelegatingExecutableMethod.java:76)
at io.micronaut.scheduling.processor.ScheduledMethodProcessor.lambda$process$5(ScheduledMethodProcessor.java:125)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
[7624.677s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
[03:37:21.925] WARN i.m.caffeine.cache.BoundedLocalCache - Exception thrown when submitting maintenance task
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at java.base/java.util.concurrent.ForkJoinPool.createWorker(ForkJoinPool.java:1329)
at java.base/java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1352)
at java.base/java.util.concurrent.ForkJoinPool.signalWork(ForkJoinPool.java:1476)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.push(ForkJoinPool.java:852)
at java.base/java.util.concurrent.ForkJoinPool.externalSubmit(ForkJoinPool.java:1919)
at java.base/java.util.concurrent.ForkJoinPool.execute(ForkJoinPool.java:2453)
at io.micronaut.caffeine.cache.BoundedLocalCache.scheduleDrainBuffers(BoundedLocalCache.java:1419)
at io.micronaut.caffeine.cache.BoundedLocalCache.performCleanUp(BoundedLocalCache.java:1452)
at io.micronaut.caffeine.cache.BoundedLocalCache$PerformCleanupTask.run(BoundedLocalCache.java:3328)
at io.micronaut.caffeine.cache.BoundedLocalCache$PerformCleanupTask.exec(BoundedLocalCache.java:3315)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
[9605.137s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
[04:10:22.372] ERROR i.m.s.DefaultTaskExceptionHandler - Error invoking scheduled task for bean [io.micronaut.management.health.monitor.HealthMonitorTask@4e56d6cb] unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:937)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1354)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
at io.micronaut.scheduling.instrument.InstrumentedExecutorService.submit(InstrumentedExecutorService.java:89)
at io.micronaut.core.async.publisher.AsyncSingleResultPublisher$ExecutorServiceSubscription.request(AsyncSingleResultPublisher.java:98)
at io.reactivex.internal.operators.flowable.FlowableFlatMap$InnerSubscriber.onSubscribe(FlowableFlatMap.java:656)
at io.micronaut.core.async.publisher.AsyncSingleResultPublisher.subscribe(AsyncSingleResultPublisher.java:59)
at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onNext(FlowableFlatMap.java:163)
at io.reactivex.internal.operators.flowable.FlowableFromIterable$IteratorSubscription.slowPath(FlowableFromIterable.java:236)
at io.reactivex.internal.operators.flowable.FlowableFromIterable$BaseRangeSubscription.request(FlowableFromIterable.java:124)
at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onSubscribe(FlowableFlatMap.java:117)
at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribe(FlowableFromIterable.java:69)
at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribeActual(FlowableFromIterable.java:47)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:53)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableFilter.subscribeActual(FlowableFilter.java:37)
at io.reactivex.Flowable.subscribe(Flowable.java:14918)
at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
at io.reactivex.Maybe.subscribe(Maybe.java:4290)
at io.micronaut.management.health.monitor.HealthMonitorTask.monitor(HealthMonitorTask.java:98)
at io.micronaut.management.health.monitor.$HealthMonitorTaskDefinition$$exec1.invokeInternal(Unknown Source)
at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
at io.micronaut.inject.DelegatingExecutableMethod.invoke(DelegatingExecutableMethod.java:76)
at io.micronaut.scheduling.processor.ScheduledMethodProcessor.lambda$process$5(ScheduledMethodProcessor.java:125)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

  2. </details>

huangapple
  • 本文由 发表于 2020年6月5日 22:04:45
  • 转载请务必保留本文链接：https://java.coder-hub.com/62217186.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定