Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-20301

ObjectQueue handles is not syncronized causing: ConcurrentModificationException

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Medium Medium
    • 3.4-m1, 3.4
    • None
    • None

      Original Description

      Method: retrieveObjectsIndividuallyAndLogFailures in ObjectQueue.java:

      the 'handles' attribute is not synced and we have reported cases that causing the DefaultObjectQueueWorker (ie indexing threads to stop working because of this. (ie indexing grinds to a halt as more and more index threads a terminated)

      Looking at at the log files in the case:

      untitled text 3:4169:  Exception in thread "Indexer: 10" java.util.ConcurrentModificationException
      untitled text 3:4181:  Exception in thread "Indexer: 9" java.util.ConcurrentModificationException
      untitled text 3:4201:  Exception in thread "Indexer: 4" java.util.ConcurrentModificationException
      untitled text 3:4289:  Exception in thread "Indexer: 8" java.util.ConcurrentModificationException
      untitled text 3:4306:  Exception in thread "Indexer: 7" java.util.ConcurrentModificationException
      untitled text 3:4329:  Exception in thread "Indexer: 1" java.util.ConcurrentModificationException
      untitled text 3:4401:  Exception in thread "Indexer: 5" java.util.ConcurrentModificationException
      untitled text 3:4838:  Exception in thread "Indexer: 2" java.util.ConcurrentModificationException
      untitled text 3:4863:  "Indexer: 6" daemon prio=10 tid=0x0000002bb5d2fc00 nid=0x508d runnable [0x0000000045f86000..0x0000000045f87c30]
      untitled text 3:5850:  "Indexer: 6" daemon prio=10 tid=0x0000002bb5d2fc00 nid=0x508d runnable [0x0000000045f86000..0x0000000045f87c30]
      untitled text 3:6601:  "Indexer: 6" daemon prio=10 tid=0x0000002bb5d2fc00 nid=0x508d runnable [0x0000000045f86000..0x0000000045f87c30]
      untitled text 3:7362:  "Indexer: 6" daemon prio=10 tid=0x0000002bb5d2fc00 nid=0x508d runnable [0x0000000045f86000..0x0000000045f87c30]
      untitled text 3:8127:  "Indexer: 6" daemon prio=10 tid=0x0000002bb5d2fc00 nid=0x508d runnable [0x0000000045f86000..0x0000000045f87c30]
      
      
      • all threads experience this ConcurrentModEx...and when a threaddump is peformed by the customer, there is only one thread left.

      The solution may be to sync the handles, as we expect this happens rarely (as it is only called in line 68 catch clause: (HibernateObjectRetrievalFailureException) and therefore less likely to affect the indexing performance.

      For consistency: sync handles is is also in Pop(), we should do the same here?

      The fixed code is this:

      
         private Collection retrieveObjectsIndividuallyAndLogFailures()
          {
              List objects = new ArrayList(handles.size());
              synchronized (handles) {
                  for (HibernateHandle handle : handles)
                  {
                      try
                      {
                          objects.add(anyTypeDao.findByHandle(handle));
                      }
                      catch (HibernateObjectRetrievalFailureException e)
                      {
                          if(log.isDebugEnabled()) {
                              log.debug("Unable to retrieve single object: " + e.getMessage(), e);
                          }
                          else {
                              log.warn("Unable to retrieve single object: " + e.getMessage());
                          }
                      }
      
                  }
      
      

      hope this helps.

      Workaround

      Attached is a single class patch for this issue. To apply it, download this file in <CONFLUENCE HOME>/WEB-INF/classes/com/atlassian/confluence/search/lucene/ (creating any missing directories as appropriate) and restart Confluence. This patch is (currently) required to resolve this issue for all versions of Confluence prior to 3.4.

              richatkins Richard Atkins
              vchoy Vincent Choy (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: