Uploaded image for project: 'Jira Software Data Center'
  1. Jira Software Data Center
  2. JSWSERVER-20691

Lexorank affects issue creation time

XMLWordPrintable

    • 8
    • 51
    • Severity 2 - Major
    • 71
    • Hide
      Atlassian Update – 14 February 2025

      Dear Customers,

      Our engineering team dedicated considerable time and effort to investigate the issue you reported.
      Fixing the problem would require extensive architectural changes to Jira Software which is not planned in the near future, that is why we have decided not to pursue a fix for this issue.

      We understand that our decision may be disappointing however we ask you to understand that this investment does not fall into the Data Center strategy and priorities.

      As a potential workaround we recommend the following:

      • Rebalance the rank and then rebuild all indices of the AO_60DB71_LEXORANK table
      • Spread issue creation throughout the day, to avoid high contention at the end of the rank
      • If possible, split work across different Jira instances

      On our side we will update documentation to clarify the inconveniences related to issue creation time affected by LexoRank.

      Best regards

      Dino Costantini
      Principal Software Engineer

      Show
      Atlassian Update – 14 February 2025 Dear Customers, Our engineering team dedicated considerable time and effort to investigate the issue you reported. Fixing the problem would require extensive architectural changes to Jira Software which is not planned in the near future, that is why we have decided not to pursue a fix for this issue. We understand that our decision may be disappointing however we ask you to understand that this investment does not fall into the Data Center strategy and priorities. As a potential workaround we recommend the following: Rebalance the rank and then rebuild all indices of the AO_60DB71_LEXORANK table Spread issue creation throughout the day, to avoid high contention at the end of the rank If possible, split work across different Jira instances On our side we will update documentation to clarify the inconveniences related to issue creation time affected by LexoRank. Best regards Dino Costantini Principal Software Engineer

      Generating the initial lexorank can seriously affect issue creation time.

      This problem can be observed in 2 ways:

      1. contention on lexorank lock when initialising rank when creating issue
      2. some nodes are never able to get the lexorank lock and create the lexorank initial value

      Problem 1

      This problem will manifests itself by increased issue creation time. 

      In logs we need to enable DEBUG logging on com.atlassian.greenhopper.service.lexorank.

      The failed locks should show up in the atlassian-greenhopper.log log.

      DEBUG Failed to acquire a lock on the max marker row and previous row for rank field

      The indexing stats (indexing-stats) should also show the Rank custom field as expensive - note the max value as the lower value of the average may be "improved" by the updates which will not trigger the expensive Lexorank initialization.

      cat ./atlassian-jira.log | grep indexing-stats | grep total

INFO      [c.a.j.issue.index.IndexingStatsManager] [indexing-stats] Top 10 addIndex total... {order:1, name:customfield_15080 (Rank), isKnown:false, addIndex: {sum:1728649386ms, avg:1765.5ms, max:7199ms, count:979152} ...
      • Note: avg: 1765.5ms, max: 7199ms

      Problem 2 

      This problem will manifests itself by increased issue creation time in the range of 18-20sec.

      Lexorank is unable to get the lexorank lock (and create the lexorank initial value) and the issue is indexed with Lexorank.MAX. 

      This will be seen in the atlassian-greenhopper.log log as warnings:

      WARN [greenhopper.customfield.lexorank.LexoRankCFType] Unable to retrieve rank for field [10005] and issue [2010933]
WARN [greenhopper.customfield.lexorank.LexoRankCFType] gh.lexorank.service.error.retrytimeout
WARN [greenhopper.customfield.lexorank.LexoRankIndexer] Could not retrieve LexoRank value for issue[id=2010933]. Indexing max LexoRank value instead.

      Notes

      • Individual ranking operation has 1500 ms timeout for rank lock. Due to DB contetion and slow operation, issue ranking operation fails to save new rank since rank lock expire.
      • See KB for more details regarding Lexorank: Troubleshooting New Ranking System Issues

      Workaround

      Currently there is no known workaround for this behavior. A workaround will be added here when available

              a9ec86424806 Dino Costantini
              mswinarski Maciej Swinarski (Inactive)
              Votes:
              37 Vote for this issue
              Watchers:
              52 Start watching this issue

                Created:
                Updated:
                Resolved: