Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-58567

Logs flood excessively with "Queue Overflow happened" WARNs when Confluence has a performance problem

XMLWordPrintable

      Issue Summary

      We are seeing that in many instances of Confluence performance issues, the logs are being spammed with the following types of messages:

      2019-05-31 10:47:09,595 WARN [http-nio-8180-exec-175] [atlassian.confluence.event.MonitorableCallerRunsPolicy] Queue Overflow happened: Runnable: [Invoker: com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimingListenerHandler$1$1@540aafe5; Event: com.atlassian.confluence.impl.content.render.xhtml.analytics.MarshallerMetricsAnalyticsEvent@16ee3cc4]; Executor: [java.util.concurrent.ThreadPoolExecutor@147ade05[Running, pool size = 6, active threads = 6, queued tasks = 512, completed tasks = 6346603]]
 -- url: /display/ds/Managing+a+Space | page: 1023213894 | traceId: ac250fe443e52573 | userName: aa0000 | referer: http://my.wiki.com/display/ds/Navigation+Page | action: viewpage
2019-05-31 10:47:09,595 WARN [http-nio-8180-exec-175] [atlassian.confluence.event.MonitorableCallerRunsPolicy] Queue Overflow happened: Runnable: [Invoker: com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimingListenerHandler$1$1@48c28143; Event: com.atlassian.confluence.impl.content.render.xhtml.analytics.MarshallerMetricsAnalyticsEvent@16ee3cc4]; Executor: [java.util.concurrent.ThreadPoolExecutor@147ade05[Running, pool size = 6, active threads = 6, queued tasks = 512, completed tasks = 6346603]]
 -- url: /display/ds/Managing+a+Space | page: 1023213894 | traceId: ac250fe443e52573 | userName: aa0000 | referer: http://my.wiki.com/display/ds/Navigation+Page | action: viewpage
2019-05-31 10:47:09,016 WARN [http-nio-8180-exec-191] [atlassian.confluence.event.MonitorableCallerRunsPolicy] Queue Overflow happened: Runnable: [Invoker: com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimingListenerHandler$1$1@48c28143; Event: com.atlassian.confluence.impl.profiling.DecoratorTimings$DecoratorTimingEvent@47e1074c]; Executor: [java.util.concurrent.ThreadPoolExecutor@147ade05[Running, pool size = 6, active threads = 6, queued tasks = 512, completed tasks = 6346502]]
 -- url: /display/ds/Navigation+Page | traceId: dec3b0debc24e67d | userName: aa0000 | referer: http://my.wiki.com/display/ds/How+to+Layout+a+Wiki+Space

      Steps to Reproduce

      1. Generate a flood of asynchronous calls to Confluence to simulate HTTP thread contention in the application

      Expected Results

      The logs will not be flooded with these messages and other potentially important information related to the root of the problem may be found in the log files

      Actual Results

      This message can be logged thousands of times in quick succession, leaving the admin with a useless log file that doesn't aid in RCA

      Workaround

      The current workaround is to extend the default logging so that more logs are kept in archive, in order to give administrators a greater window to capture the relevant logs. This can be done my modifying the following two parameters in <confluence_install>/confluence/WEB-INF/classes/log4j.properties, which influences size of log files and number of logs kept in archive, respectively:

      log4j.appender.confluencelog.MaxFileSize=20480KB
log4j.appender.confluencelog.MaxBackupIndex=5

      Note however that increasing the logs kept will impact the disk footprint and size of Support Zips generated.

      Extra Workaround:
      You can also choose to get rid of the noisy logging by going to Logging and Profiling and setting below class as FATAL:

      com.atlassian.confluence.event.MonitorableCallerRunsPolicy

              Unassigned Unassigned
              nlopez@atlassian.com Nancy Lopez
              Votes:
              12 Vote for this issue
              Watchers:
              25 Start watching this issue

                Created:
                Updated:
                Resolved: