Issue Summary

If there is an interruption to Jira’s connection to the database while a cluster lock is trying to be updated/cleared, the system will try three times, 100ms apart and then fail without trying further. This can lead to stale entries being left behind in the clusterlockstatus table where the locked_by_node attribute is still set to a node without being cleaned up. Normally, when a node 'unlocks' a cluster lock, it doesn't actually delete the entry from the database, it just sets the locked_by_node attribute to null. So the entries will still be in the database, but any node will be allowed to take it.

These leftover entries in the database are unlikely to cause harm. Any lock that hasn't been updated for more than five minutes is treated as stale and can simply be taken over by whichever node needs it next, so nothing is blocked, and clearing them out isn't urgent. They'll continue to be listed in the database as held until each one is next used, even though they're actually free. As a result, these stale entries will likely appear in Jira's health check reports when they occur.

Please note: if you are experiencing functional issues in Jira when stale cluster locks occur, please contact support to have your situation evaluated.
While the 'Affects Version/s" is set to 11.3.10, and 10.3.24, this behaviour is present in all currently supported versions of Jira and was not practical to list all versions. Therefore, the latest LTS releases are listed instead.

Steps to Reproduce

Reproduction of the exact scenario needs a break in connection of Jira to the database:

• Jira DC connected to database (more possible with pgpool/HA setup at DB)
• Database connection is teminated
• Database takes time time to come up (more than 5 seconds)
• Jira connects back to the database
• Stale lock entries remain in the clusterlockstatus table that Jira could not clear automatically.

Expected Results

Stale (older than five minutes) entries in the clusterlockstatus table are cleared (or ‘unlocked’) so that health check reports do not display entries.

Actual Results

Stale entries are kept and will end up staying in the database indefinitely.

Workaround

A reboot of the node holding the lock should ‘unlock’ the stale locks and clear any health checks.