Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-61338

Workbox Notification will timeout on first attempt, and shows blank page and System Error message(Regression)

XMLWordPrintable

      The fix for this bug has been released to our Long Term Support release.

      The fix for this bug is now available in the latest release of Confluence 7.13 and 7.19

      Notice: This issue is regression of CONFSERVER-55134.

      I got also the same problem with version 7.4.7:

      2021-02-24 01:49:08,213 ERROR [http-nio-8090-exec-6] [[Standalone].[localhost].[/].[servlet-module-container-servlet]] log Servlet.service() for servlet [servlet-module-container-servlet] in context with path [] threw exception
java.lang.RuntimeException: java.util.concurrent.TimeoutException
        at com.google.common.base.Throwables.propagate(Throwables.java:241)
        at com.atlassian.mywork.host.util.concurrent.LockedTaskExecutor.executeUnderLock(LockedTaskExecutor.java:106)
        at com.atlassian.mywork.host.service.LocalRegistrationServiceImpl.getLastModified(LocalRegistrationServiceImpl.java:192)

      2021-02-24 01:49:13,226 ERROR [AtlassianEvent::CustomizableThreadFactory-2] [renderer.internal.http.HttpClientFetcher] fetch Unable to perform a request to: http://MacBook-Pro.local:41252/rest/gadgets/1.0/g/messagebundl
e/und/gadget.common%2Cgadget.confluence
org.apache.http.conn.ConnectTimeoutException: Connect to MacBook-Pro.local:41252 [MacBook-Pro.local/192.168.1.2] failed: connect timed out
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:394)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

      ==== Quoted from CONFSERVER-55134 ====

      Summary

      Workbox Notification does not work and shows empty page on first attempt following a Confluence restart. It will work and able to retrieve information on the next attempts/ page refresh.

      Steps to Reproduce

      1. On a fresh installation of Confluence,
      2. Click on the Workbox Notification icon on the top right corner

      Expected Results

      The workbox notification loads correctly

      Actual Results

      1. The workbox notification shows blank page:
      2. After a while, at times, a System Error appears in the page after a page refresh

      The following appears in the atlassian-confluence.log when the page is blank

      2018-03-07 23:20:57,928 ERROR [http-nio-8090-exec-5] [atlassian.confluence.event.ConfluenceEventDispatcher] run There was an exception thrown trying to dispatch event [com.atlassian.mywork.host.event.BeforeCountNewNotificationsEvent@31bba1bf] from the invoker [com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimingListenerHandler$1$1@7794c789]
 -- referer: http://localhost:8090/dashboard.action | url: /rest/mywork/latest/status/notification/count | traceId: 2877b0225ae534da | userName: admin
java.lang.RuntimeException: service proxy has been destroyed. Listener: com.atlassian.mywork.host.service.ClientServiceImpl event: com.atlassian.mywork.host.event.BeforeCountNewNotificationsEvent

      Eventually, it will timeout with the following:

      2018-03-07 23:22:44,209 ERROR [http-nio-8090-exec-5] [[Standalone].[localhost].[/].[servlet-module-container-servlet]] log Servlet.service() for servlet [servlet-module-container-servlet] in context with path [] threw exception
java.lang.RuntimeException: java.util.concurrent.TimeoutException
	at com.google.common.base.Throwables.propagate(Throwables.java:160)

      The following appears in the log when seeing the System Error:

      2018-03-08 18:43:05,452 ERROR [http-nio-8090-exec-6] [host.util.concurrent.LockedTaskExecutor] executeUnderLock Timed out waiting for lock before getting last modified date
 -- referer: http://localhost:8090/dashboard.action | url: /plugins/servlet/notifications-miniview | traceId: 56ebeb370dca72ad | userName: admin
java.util.concurrent.TimeoutException
	at com.atlassian.mywork.host.util.concurrent.LockedTaskExecutor.tryExecuteUnderLock(LockedTaskExecutor.java:66)
	at com.atlassian.mywork.host.util.concurrent.LockedTaskExecutor.executeUnderLock(LockedTaskExecutor.java:116)
	at com.atlassian.mywork.host.service.LocalRegistrationServiceImpl.getLastModified(LocalRegistrationServiceImpl.java:205)
	at com.atlassian.mywork.host.service.LocalRegistrationServiceImpl.getCacheValue(LocalRegistrationServiceImpl.java:244)
	at com.atlassian.mywork.host.servlet.ServletRenderer.renderWithAnchor(ServletRenderer.java:79)
	at com.atlassian.mywork.host.servlet.ServletRenderer.renderWithAnchor(ServletRenderer.java:46)
	at com.atlassian.mywork.host.servlet.NotificationsMiniviewServlet.doGet(NotificationsMiniviewServlet.java:23)

      Workaround

      Refresh the page a couple of times

      Notes

      The issue happens to every user, during the first time accessing Workbox Notification following a restart.

      Other than restarting Confluence, this can be easily replicable by accessing Workbox Notification for the first time following enabling a previously disabled Workbox plugins::

      1. Workbox - Common Plugin
      2. Workbox - Confluence Provider Plugin
      3. Workbox - Host Plugin

      Confluence Data Center

      It is possible that the issue doesn't go away even after many refresh in a Confluence Data Center Cluster. The thread that tries to load the notification box stays the same like below until it reaches the timeout:

      "http-nio2-8090-exec-20 url: /plugins/servlet/notifications-miniview; user: admin" #xxx daemon prio=5 os_prio=0 cpu=xxx.xxms elapsed=xxx.xxs tid=xx00007f871001xxxx nid=0xxxxc waiting on condition  [0x00007f85e74exxxx]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.14.1/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.14.1/LockSupport.java:323)
	at com.hazelcast.spi.impl.AbstractInvocationFuture.get(AbstractInvocationFuture.java:161)
	at com.hazelcast.concurrent.lock.LockProxySupport.tryLock(LockProxySupport.java:138)
	at com.hazelcast.concurrent.lock.LockProxySupport.tryLock(LockProxySupport.java:127)
	at com.hazelcast.map.impl.proxy.MapProxyImpl.tryLock(MapProxyImpl.java:474)
...

      Note that the thread is now hazelcast-related

      In this case, a full restart of the entire Confluence Cluster is required:

      1. Stop all Confluence nodes
      2. Start Confluence node one at a time

        1. confluence-v7.4.7-CONFSERVER-55134.png
          confluence-v7.4.7-CONFSERVER-55134.png
          715 kB

              2ac182705928 Ragan Martinez
              ff3967e4ed5f Ryoji Takata (Inactive)
              Votes:
              17 Vote for this issue
              Watchers:
              39 Start watching this issue

                Created:
                Updated:
                Resolved: