Issue Summary

On Confluence Data Center 9.2.26 (build 9117), the Clustering admin page fails with a NullPointerException in MonitoringError.<init> when a node's local Hazelcast instance is inactive but the node is still serving requests. The admin gets the generic error page instead of a message saying the node's data couldn't be loaded.

Steps to Reproduce

1. Run a 2-node Confluence Data Center cluster.
2. Get one node into a state where its local Hazelcast instance is inactive but Confluence and Tomcat keep serving requests. That node's application log shows HazelcastInstanceNotActiveException: HazelcastInstance[[<node-ip>]:5801] is not active!
3. On the affected node, log in as a system admin and go to General Configuration > Clustering.
4. Open that node's runtime information. The page calls:
   /rest/atlassian-cluster-monitoring/cluster/suppliers/data/com.atlassian.cluster.monitoring.cluster-monitoring-plugin/runtime-information/<node-id>

Expected Results

The Clustering page loads and shows that the node's runtime information couldn't be retrieved.

Actual Results

The REST call returns 500 and the admin gets the generic error page. The error handling fails too: an NPE is thrown inside the MonitoringError constructor at line 19. The stack trace doesn't show which value was null.

The below exception is thrown in the atlassian-confluence.log file:

<timestamp> ERROR [http-nio2-8080-exec-<n> url: /rest/atlassian-cluster-monitoring/.../runtime-information/<node-id>; user: <admin-user>] ...
java.lang.NullPointerException
    at com.atlassian.cluster.monitoring.spi.model.MonitoringError.<init>(MonitoringError.java:19)
    at com.atlassian.confluence.cluster.hazelcast.monitoring.HazelcastClusterMonitoring.getData(HazelcastClusterMonitoring.java:103)

Impact:  Administrators use the Clustering page to check the status of each node. When a node's Hazelcast instance is inactive, the page shows the generic error page instead of showing that the node's data couldn't be loaded. So the page fails in exactly the situation it's meant to help diagnose.

Related observation (not a second instance of this bug): on the other node, while its Hazelcast instance was inactive, a request failed and Tomcat also failed to render /500page.jsp. This was logged in catalina.out:

SEVERE [http-nio2-8080-exec-<n>] org.apache.catalina.core.StandardHostValve.custom Exception Processing [ErrorPage[errorCode=500, location=/500page.jsp]]
        org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction
...
Caused by: com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!

Workaround

Currently there is no known workaround for this behavior. A workaround will be added here when available