In a JIRA Data Center Environment, ClassNotFoundException can be seen in the logs due to an installed plugin thread initializing JIRA Data Center cache with its own classloader.
- JIRA Data Center
- Reproduced with JIRA Data Center 7.1.x with Structure Plugin 3.1
- Setup a JIRA Data Center with at least 2 nodes.
- Find / build a plug-in which starts a thread in a constructor / plugin event / module event listener.
- That thread inside of the 3rd party plug-in / custom made plugin should be the first who initialises the cache. For example the thread can call com.atlassian.jira.user.ApplicationUsers.byKey to cause cache initialisation.
- NOT FOR PRODUCTION USE. You can use the attached jira-cache-accessor-plugin.jar as a demo how to init the cache with the wrong classloader.
- Observe the logs during JIRA startup. Alternative - see the logs on cache synchronization.
JIRA webapp classloader is used to initialize and access the cache, and there are no errors with replicating the cache to other nodes
- Cache is initialized with the plugin's class loader, rather than the correct class loader for the code.
In the sample plug-in we can see that see now which class loader used to init the EagerOfBizUserCache. I.e. initialized with class loader: BundleDelegatingClassLoader. To see that I've modified EagerOfBizUserCache to log which class loader is used to initialize the cache.
- Errors similar to the following are thrown in the application logs during cache replication to other nodes. Note that this errors will be show at "healthy node" and actual problem is at other node(s).
- Plugin developers may consider wrapping such calls with a class loader switching code, to ensure that the right class loader is used. For example, in a case of calling ApplicationUser's static methods, the class loader switch may be as easy as:
- Disable the plugin that triggered the cache problem: Structure or other
- Restart node(s) not having mentioned error
- Enable plugin again
If the workaround above does not work on your case, try the following:
- Disable the plugin triggering the cache problem (Structure, ScriptRunner or other).
- Remove node1 from the load balancer and restart JIRA on that node.
- When node1 is back up, re-add it to the load balancer and restart node2.
- Repeat steps 2-3 for the other nodes.
- After all nodes have been restarted, enable the plugin causing the issue again.
Almworks has released new version 3.3.0.jira7 which has the fix.