Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-20878

Manual re-indexing process hangs at 99%

    XMLWordPrintable

Details

    Description

      Bug background

      The re-indexing process might take long time to complete especially when the instance is large. Due to this, many organisations schedule this process during off peak hours, usually after 12am. However, the process hangs at 99% completion with the following error in the logs:

      2010-09-22 08:18:52,832 ERROR [Indexer: 4] [confluence.search.lucene.DefaultObjectQueueWorker] indexCollection Error indexing document
       -- url: /admin/reindex.action | userName: admin | referer: http://support.cosinex.de/admin/search-indexes.action | action: reindex
      com.atlassian.bonnie.LuceneException: java.io.IOException: cannot read directory org.apache.lucene.store.FSDirectory@/home/confluence/data-3.1/temp/lucene2506686581554376744index: list() returned null
      	at com.atlassian.confluence.search.lucene.TempIndexWriterStrategy.perform(TempIndexWriterStrategy.java:47)
      	at com.atlassian.confluence.search.lucene.tasks.TempIndexBackedIndexTaskPerformer.perform(TempIndexBackedIndexTaskPerformer.java:21)
      	at com.atlassian.confluence.search.lucene.DefaultObjectQueueWorker.indexCollection(DefaultObjectQueueWorker.java:79)
      	at com.atlassian.confluence.search.lucene.DefaultObjectQueueWorker$1.doInTransactionWithoutResult(DefaultObjectQueueWorker.java:62)
      	at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
      	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127)
      	at com.atlassian.confluence.search.lucene.DefaultObjectQueueWorker.run(DefaultObjectQueueWorker.java:51)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	at java.lang.Thread.run(Thread.java:619)
      Caused by: java.io.IOException: cannot read directory org.apache.lucene.store.FSDirectory@/home/confluence/data-3.1/temp/lucene2506686581554376744index: list() returned null
      	at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:260)
      	at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:1950)
      	at org.apache.lucene.index.IndexWriter.flushRamSegments(IndexWriter.java:1741)
      	at org.apache.lucene.index.IndexWriter.flushRamSegments(IndexWriter.java:1733)
      	at org.apache.lucene.index.IndexWriter.maybeFlushRamSegments(IndexWriter.java:1727)
      	at org.apache.lucene.index.IndexWriter.deleteDocuments(IndexWriter.java:1028)
      	at com.atlassian.confluence.search.lucene.tasks.DeleteChangeDocumentsIndexTask.perform(DeleteChangeDocumentsIndexTask.java:70)
      	at com.atlassian.confluence.search.lucene.tasks.RebuildChangeDocumentsIndexTask.perform(RebuildChangeDocumentsIndexTask.java:66)
      	at com.atlassian.bonnie.index.TempIndexWriter.perform(TempIndexWriter.java:73)
      	at com.atlassian.confluence.search.lucene.TempIndexWriterStrategy.perform(TempIndexWriterStrategy.java:43)
      	... 9 more
      
      Causes/Troubleshooting/Workaround
      1. The first step to try is to ensure that the <confluence-home>/index and <confluence-home>/temp directories have the read/write permissions
      2. Also, it's also worth to try to re-index from scratch
      3. If the above suggestions doesn't work, it's probably a good idea to check what time the re-index process takes place as there's a Quartz job called cleanTempDirectoryJob that cleans the <confluenec-home>/temp directory everyday at 4am. This job might be clashing with the indexing process which is causing the indexing process to stop at 99%
      Suggestions

      Perhaps, there should be locking mechanism that will stop other jobs that are using resources required by the indexing process

      Attachments

        Issue Links

          Activity

            People

              shaffenden Steve Haffenden (Inactive)
              sjayaraman Sashidaran Jayaraman [Atlassian]
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: