-
Bug
-
Resolution: Not a bug
-
High
-
None
-
9.0.0, 9.4.0, 9.4.3, 9.7.0
-
9
-
6
-
Severity 1 - Critical
-
45
-
-
Issue Summary
When Jira is upgraded from 8.x to 9.x and the first node starts up for the first time, it performs the database upgrade and adheres to the jira-config.properties config:
upgrade.reindex.allowed=false
However, if the node's restarted, this second startup kicks-off a full reindex regardless.
Impact
Restarting Jira right after upgrade is not uncommon and reasons, vary like:
- Admins realizing some customizations to config files were not re-applied
- Admins realizing JVM custom parameters were not re-applied
- Any other issue that didn't prevent DB upgrade tasks but called for a restart
Jira 8 behaved in a way the "upgrade.reindex.allowed=false" would bypass full reindexes on every restart, but Jira 9 enforces a healthy Index to start-up properly (either local or from the shared home, or else it full reindexes).
It seems Jira 9 interprets the config differently (maybe correctly now, and it was Jira 8 that had a "bug feature"): "upgrade.reindex.allowed" is only relevant on the startup that's actually performing the upgrade. A restart right after the upgrade's finished is no longer considered "an upgrade" and the config is thus ignored.
The call for restarts may be unpredictable and may happen in one environment more than in another, and this full reindex is getting admins off guard and compromising the maintenance schedules and outage windows, leading to unplanned downtime.
Impact aggravation
An unexpected reindex is no good, but the impact's much worsened by Jira 9 taking much longer to reindex than Jira 8 on some datasets: JRASERVER-74787.
Steps to Reproduce
- Install Jira 8.20 (2 node cluster)
- Load some data into it (through sample projects or Data Generator for Jira)
- Perform a full reindex
- Bring both nodes down
- Upgrade Jira to 9.4 in "node1" with "upgrade.reindex.allowed=false"
- Confirm in the logs no full reindex was kicked-off
- Restart Jira
- Observe the logs for full reindex messages and that Jira's locked while reindexing
(with just a few data the outage may not be perceptible thus the logs are the main evidence of full reindex)2023-02-22 13:01:25,586-0300 JiraTaskExecutionThread-1 INFO anonymous [c.a.j.util.index.CompositeIndexLifecycleManager] Reindex All COMPLETED without any errors. Total time: 4683ms. Reindex run: 4
Expected Results
Jira would refrain from kicking-off an automatic reindex and wait for the admin to start it manually through the UI (or REST API).
Actual Results
The second startup after the upgrade routines enforce a Jira 9 compatible index. If it can't find one locally, it tries for the shared-home and last, locks itself and performs a full reindex.
Workaround
- Add these two JVM flags to the startup parameters (see Setting properties and options on startup):
-Dcom.atlassian.jira.startup.allow.full.reindex=false -Dcom.atlassian.jira.startup.rebuild.local.index=false
They can be added to the setenv.sh/bat in the "JVM_SUPPORT_RECOMMENDED_ARGS" variable.
We also published a Practical guide on upgrading Jira Data Center to version 9 or 9.4 LTS with a clear mention to these configs and requested other documentation updates through JRASERVER-74957
(updated on March 14th 2023)
- is related to
-
JRASERVER-74787 Full reindex operation slow down in Jira 9.0.0 caused by fetching issue versions from database in a single thread
- Closed
-
JRASERVER-74957 Review and update the Jira 9 upgrade guides
- Closed
- mentioned in
-
Page Loading...
-
Page Loading...
-
Page Loading...
-
Page Loading...
-
Page Loading...
-
Page Loading...