-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Highest
-
Affects Version/s: 11.0.0
-
Component/s: Upgrade
-
11
-
26
-
Severity 2 - Major
-
329
Jira 11.3.12 introduces a new parameter:
-Dcom.atlassian.jira.startup.lucene.skip.index.recovery=true
Default: false.
When set to true, Jira skips Lucene index recovery entirely during startup. The index is neither validated nor recovered, and the node is allowed to start. Index replication stays paused on that node, so it will not apply index updates coming from other nodes.
How to upgrade from 10.x to 11.x using the property
- Back up your database, shared home and local home. Note that 10.x → 11.x is a major platform upgrade and cannot be done as a rolling (zero downtime) upgrade.
- Shut down all nodes in the cluster.
- Upgrade the binaries on node 1 to 11.x and add the property to that node's JVM arguments (in setenv.sh / setenv.bat, via JVM_SUPPORT_RECOMMENDED_ARGS).
- Start node 1 only. It will log a warning that index recovery was skipped, then reach a running state. The upgrade tasks execute and the build number in the database advances to 11.x.
- Run a full foreground re-index: Administration → System → Indexing → Full re-index, option "Lock Jira and rebuild index". Do not use the background re-index, it cannot repair an index with incompatible field types and will fail part way through.
- When the re-index finishes, shut down node 1, remove the property, and start node 1 again. It will now start through the normal index path.
- Before starting the remaining nodes, remove the pre-upgrade index snapshots from the shared home so that those nodes cannot restore a 10.x index. Then start nodes 2..N one at a time, without the property.
Notes
- Do not leave the property enabled in normal operation. A node running with a skipped index recovery is not in a supported state.
Thank you for your patience while we worked through this one.
Konrad Plasota
Software Engineer
Issue Summary
When upgrading Jira Data Center from version 10.x to 11.x, the application may fail to start or experience significant functional degradation (broken gadgets, missing side panels, failed webhooks) due to an index incompatibility. Specifically, Jira 11 introduces breaking changes to how subtasks and issue properties are indexed (switching from SORTED_NUMERIC to SORTED_SET). If Jira attempts to reuse or incrementally update an existing Jira 10 index, it throws an IllegalArgumentException, which subsequently blocks the UpgradeLauncher and prevents the JiraStartedEvent from firing.
Steps to Reproduce
- Install Jira Data Center version 10.3.x and create issues with subtasks.
- Perform an upgrade to Jira Data Center version 11.3.x.
- Start the Jira 11 instance using the existing Jira 10 index without performing a full re-index.
- Observe the startup logs and application behavior.
Expected Results
Jira should either automatically detect the index version mismatch and force a full re-index, or provide a clear upgrade task/documentation requiring a full re-index to prevent startup failure.
Actual Results
Jira fails to run upgrade tasks and initialize the index correctly. The below exception is thrown in the atlassian-jira.log file:
main ERROR [c.a.jira.upgrade.UpgradeLauncher] Skipping, JIRA is locked. main INFO [c.a.jira.scheduler.JiraSchedulerLauncher] JIRA Scheduler not started: JIRA startup failed. main INFO [c.a.jira.scheduler.JiraSchedulerLauncher] The following failures prevent Jira from being started: main INFO [c.a.jira.scheduler.JiraSchedulerLauncher] (Event: Level = (EventLevel: warning) , Key = (EventType: no-index) , Desc = We could not find a valid search index on the current node , Exception = Jira needs to have a valid search index after starting. During the node startup, all of the following steps failed to pass: <ul><li>Re-index missing data if a local issue index is less than 10% behind the database.</li><li>Check if Jira shared-home directory contains a recent index snapshot.</li><li>Trigger a full re-index.</li></ul><B>You need to fix this issue manually. Do not start other nodes until the indexing problem is resolved. </B>) main INFO [c.a.jira.startup.LauncherContextListener] Startup is complete. Jira is ready to serve.
IssueIndexer:thread-2 WARN [c.a.jira.index.AccumulatingResultBuilder] Indexing failed for Issue - '100000'
IssueIndexer:thread-2 WARN [c.a.jira.index.AccumulatingResultBuilder] cannot change DocValues type from SORTED_NUMERIC to SORTED_SET for field "issue_subtasks"
java.lang.IllegalArgumentException: cannot change DocValues type from SORTED_NUMERIC to SORTED_SET for field "issue_subtasks"
at org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:289)
at org.apache.lucene.index.FieldInfos$FieldNumbers.setDocValuesType(FieldInfos.java:337)
...
IssueIndexer:thread-3 WARN [c.a.jira.index.AccumulatingResultBuilder] Indexing failed for Issue - '100000' IssueIndexer:thread-3 WARN [c.a.jira.index.AccumulatingResultBuilder] cannot change DocValues type from SORTED to SORTED_SET for field "ISSUEPROP_jpo-issue-properties$baseline_start" java.lang.IllegalArgumentException: cannot change DocValues type from SORTED to SORTED_SET for field "ISSUEPROP_jpo-issue-properties$baseline_start" at org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:289) at org.apache.lucene.index.FieldInfos$FieldNumbers.setDocValuesType(FieldInfos.java:337) ...
Upgrade task banner and dashboard gadget broken:
Workaround
To resolve this, you must force Jira to perform a full re-index on startup by bypassing the local index and shared snapshots. Add the following JVM arguments to your Jira startup configuration:
-Dcom.atlassian.jira.startup.rebuild.local.index=false -Dcom.atlassian.jira.startup.pick.indexsnapshot.from.shared=false -Dcom.atlassian.jira.startup.allow.full.reindex=true
Note: Ensure only one node is started initially to perform the full re-index before joining other nodes to the cluster.
