Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-64233

Recover Issue Index is a blocking and single threaded process (is not a foreground nor background reindex)

    XMLWordPrintable

Details

    Description

      Summary

      There is a scenario where JIRA during start up detects that there is a gap between the issues in the database and the indexed issues (lucene's indexes), if that gap is lower than 24 hours the start up thread starts a a single-threaded reindex (Recover Issue Index) blocking any other threads and hence making the reindex slower and blocking any users connections to JIRA.
      It is important to note that this process only happens during the start up.

      Environment

      I reproduced it in version 6.4.13 as well as version 7.2.

      Steps to Reproduce

      1. Start up JIRA
      2. Mark log file "Recover Issue Index Test Start"
      3. Create a new project
      4. Insert 1000 issues to the new project
      5. Goto System>>Advanced>>Indexing>>Index Recovery and enable it
      6. Create a new backup for the indexes
      7. Insert 10000 new issues (please note it can be less than 10000, I used 10000 issues as it gave me more time to probe that blocks any other sessions)
      8. Stop JIRA
      9. Goto JIRA_HOME/caches/indexes and copy all directories to a temp directory so you have a backup
      10. Delete all the directories in JIRA_HOME/caches/indexes
      11. Copy the backup generated in step 6 from JIRA_HOME/export/indexsnapshots/ and unzip it to JIRA_HOME/caches/indexes
      12. Start up JIRA
      13. Mark log file "Recover Issue Index Test 2 End"
      14. Check the log files

      Expected Results

      If the start up thread kicks off a reindex and blocks any other users I would expect the process to be multi-threaded. Ideally, the customer would be able to configure what type of reindex process they would like to have in this situation by choosing either a background reindex (single threaded and nonblocking) or foreground re-index (multi-threaded, but blocking) using a property. I have created a separate feature request for this.

      Actual Results

      The start up thread kicks off a reindex process that blocks any other threads and is single-threaded.

      Workaround

      There is no workaround.

      Additional Information

      Start of the process in the log file

      2017-02-11 13:00:31,540 localhost-startStop-1 INFO [atlassian.jira.scheduler.JiraSchedulerLauncher] Starting the JIRA Scheduler....
      2017-02-11 13:00:31,596 localhost-startStop-1 INFO [atlassian.jira.scheduler.JiraSchedulerLauncher] JIRA Scheduler started.
      2017-02-11 13:00:33,215 localhost-startStop-1 INFO [jira.index.ha.DefaultIndexRecoveryManager] Latest index date: {2017-02-11 12:29:49}, Latest DB date: {2017-02-11 12:53:06}
      2017-02-11 13:00:33,220 localhost-startStop-1 INFO [atlassian.jira.startup.IndexRecoveryLauncher] Recover Issue Index - start
      2017-02-11 13:00:33,692 localhost-startStop-1 DEBUG [jira.issue.index.DefaultIssueDocumentFactory] Indexing issue: STP-14998
      2017-02-11 13:00:33,964 localhost-startStop-1 DEBUG [jira.issue.index.DefaultIssueDocumentFactory] Indexing issue: STP-14997
      2017-02-11 13:00:34,002 localhost-startStop-1 DEBUG [jira.issue.index.DefaultIssueDocumentFactory] Indexing issue: STP-1499
      

      End of the process in the log files

      2017-02-11 13:04:10,745 localhost-startStop-1 DEBUG [jira.issue.index.DefaultIssueDocumentFactory] Indexing issue: STP-998
      2017-02-11 13:04:10,755 localhost-startStop-1 DEBUG [jira.issue.index.DefaultIssueDocumentFactory] Indexing issue: STP-997
      2017-02-11 13:04:10,772 localhost-startStop-1 DEBUG [jira.issue.index.DefaultIssueDocumentFactory] Indexing issue: STP-996
      2017-02-11 13:04:11,453 localhost-startStop-1 INFO [atlassian.jira.startup.IndexRecoveryLauncher] Recover Issue Index - end
      2017-02-11 13:04:11,457 localhost-startStop-1 INFO [atlassian.jira.startup.LauncherContextListener] Memory Usage:
      ---------------------------------------------------------------------------------
       Heap memory : Used: 228 MiB. Committed: 371 MiB. Max: 742 MiB
       Non-heap memory : Used: 209 MiB. Committed: 228 MiB. Max: 1264 MiB
      ---------------------------------------------------------------------------------
       TOTAL : Used: 436 MiB. Committed: 600 MiB. Max: 2006 MiB
      ---------------------------------------------------------------------------------
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ecasuscelli Esteban Casuscelli
              Votes:
              5 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: