Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-66557

ClusterMessageHandlerServiceThread can stop checking messages if Throwable is encountered

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Low
    • None
    • 7.2.12, 7.6.2
    • Data Center - Other

    Description

      Summary

      ClusterMessageHandlerServiceThread is capable of entering a state where it is no longer checking messages. This can cause inconsistency between JIRA Data Center Nodes.

      Environment

      • JIRA Data Center

      Expected Results

      • ClusterMessageHandlerServiceThread always checks messages every three seconds.

      Expected stacktrace while waiting:

      "ClusterMessageHandlerServiceThread:thread-1" #109 prio=5 os_prio=0 tid=0x00007ff544016800 nid=0x7ee7 waiting on condition [0x00007ff508581000]
         java.lang.Thread.State: TIMED_WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x00000000d18db240> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
              at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
              at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      Actual Results

      • ClusterMessageHandlerServiceThread enters a state where it no longer checks messages every three seconds.
        • Thread is no longer in TIMED_WAITING and is stuck in WAITING

      Unexpected stacktrace. No longer checking messages:

      • Affected threads would show this stacktrace and be stuck in this state without changing.
      "ClusterMessageHandlerServiceThread:thread-1" prio=5 tid=0x000000000000006c nid=0 waiting on condition 
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for <0x0000000020f2b558> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:748)
      

      Notes

      • Developer review "found that the ClusterMessageHandlerServiceThread can go into WAITING state if any of the ClusterMessage consumers throw a Throwable exception."

      Workaround

      • No workaround is available.
      • ClusterMessageHandlerServiceThread that is no longer checking messages requires node to be restarted.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              tevans Tim Evans (Inactive)
              Votes:
              3 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: