Details
-
Suggestion
-
Resolution: Fixed
-
None
Description
When a system has a lot of missing rank values for issues or none at all, a foreground reIndex will trigger rankInitially for a lot of issues. This type of rank operation will rank the issue to the bottom of the ordering, which requires the bottom 2 rows to be locked before a value can be assigned. Foreground reIndex uses up to 10 threads so this will cause a lot of contention to acquire locks on those 2 bottom rows.
The entries below are visible in the logs:
(...) 2014-06-06 10:52:09,857 IssueIndexer:thread-7 WARN jirauser 778x220x1 10qtvrt 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex.jspa [greenhopper.customfield.lexorank.LexoRankCFType] Unable to retrieve rank for field [10801] and issue [69426] 2014-06-06 10:52:09,857 IssueIndexer:thread-7 WARN jirauser 778x220x1 10qtvrt 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex.jspa [greenhopper.customfield.lexorank.LexoRankCFType] gh.lexorank.service.error.retrytimeout 2014-06-06 10:52:09,857 IssueIndexer:thread-7 WARN jirauser 778x220x1 10qtvrt 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex.jspa [greenhopper.customfield.lexorank.LexoRankIndexer] Could not retrieve LexoRank value for issue[id=69426]. Indexing max LexoRank value instead. 2014-06-06 10:52:10,748 IssueIndexer:thread-3 WARN jirauser 778x220x1 10qtvrt 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex.jspa [greenhopper.customfield.lexorank.LexoRankCFType] Unable to retrieve rank for field [10801] and issue [69420] 2014-06-06 10:52:10,748 IssueIndexer:thread-3 WARN jirauser 778x220x1 10qtvrt 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex.jspa [greenhopper.customfield.lexorank.LexoRankCFType] gh.lexorank.service.error.retrytimeout 2014-06-06 10:52:10,748 IssueIndexer:thread-3 WARN jirauser 778x220x1 10qtvrt 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex.jspa [greenhopper.customfield.lexorank.LexoRankIndexer] Could not retrieve LexoRank value for issue[id=69420]. Indexing max LexoRank value instead. (...)
This can impact the instance by causing excessive reindexing times. For example an instance with the following stats has taken 6 hours to index and is only at 26%:
___ Database Statistics ____________________
Issues : 397982
Projects : 218
Custom Fields : 165
Workflows : 120
Users : 18587
Groups : 2805
Attachments : 163215
Comments : 1567415
We should introduce a JVM lock on the rankInitially part of the rank operation reduce this expected contention.
Workaround
- Stop JIRA to cancel the index.
- Backup the database.
- Start JIRA.
- Unlock the rank fields as per How to unlock a Locked field.
- Rename the rank fields to something else, for example Rank (LexoRank).
- Rename the old (Obsolete) fields to their original name.
- Uninstall JIRA Agile.
- Stop JIRA.
- Roll-back the JIRA Agile version in the database with the following SQL:
UPDATE propertynumber SET propertyvalue = 42 WHERE id = (SELECT id FROM propertyentry WHERE property_key = 'GreenHopper.Upgrade.Latest.Upgraded.Version'); UPDATE propertystring SET propertyvalue = '42' WHERE id = (SELECT id FROM propertyentry WHERE property_key = 'com.pyxis.greenhopper.jira:build');
- Start JIRA.
- Install the previous version of JIRA Agile 6.3.13.1.
- Reindex JIRA.