Details
-
Bug
-
Resolution: Unresolved
-
Low
-
None
-
9.4.2
-
9.04
-
4
-
Severity 2 - Major
-
6
-
Description
Problem
In a JIRA Data Center Environment, ClassNotFoundException can be seen in the logs due to an installed plugin thread initializing JIRA Data Center cache with its own classloader. The error was fixed previously in JRASERVER-62071 - Installed plugins initializing Data Center cache with wrong class loader but resurfaced in recent 9.X releases
2023-03-23 05:50:45,791+0100 localq-reader-74 WARN [c.a.j.c.distribution.localq.LocalQCacheOpReader] [LOCALQ] [VIA-COPY] Runtime exception: UnrecoverableFailure occurred when processing: LocalQCacheOp{cacheName='com.atlassian.jira.crowd.embedded.ofbiz.EagerOfBizUse rCache.userCache', action=PUT, key={10000,YY_XXX}, value == null ? false, replicatePutsViaCopy=true, creationTimeInMillis=1679547045779} from cache replication queue: [queueId=queue_node4_4_9e22b2ee283109ab44b3ddeb56f9ed7a_put, queuePath=/var/atlassian/application-data/ jira/localq/queue_node4_4_9e22b2ee283109ab44b3ddeb56f9ed7a_put], failuresCount: 1/1, error: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: java.lang.ClassNotFoundException: net.sf.ehcache.Element not found by com.package.XXXX [234] (no security manager: RMI class loader disabled) 2023-03-23 05:50:45,791+0100 localq-reader-74 ERROR [c.a.j.c.distribution.localq.LocalQCacheOpReader] [LOCALQ] [VIA-COPY] Abandoning sending: LocalQCacheOp{cacheName='com.atlassian.jira.crowd.embedded.ofbiz.EagerOfBizUserCache.userCache', action=PUT, key={10000,u_m oze}, value == null ? false, replicatePutsViaCopy=true, creationTimeInMillis=1679547045779} from cache replication queue: [queueId=queue_node4_4_9e22b2ee283109ab44b3ddeb56f9ed7a_put, queuePath=/var/atlassian/application-data/jira/localq/queue_node4_4_9e22b2ee283109ab44b . . . Caused by: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: java.lang.ClassNotFoundException: net.sf.ehcache.Element not found by com.package.XXX [234] (no security manager: RMI class loader disabled) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:389) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:303) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:279) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227) at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179) at com.sun.proxy.$Proxy33.put(Unknown Source) at com.atlassian.jira.cluster.distribution.localq.rmi.LocalQCacheOpRMISender.lambda$send$2(LocalQCacheOpRMISender.java:69) at com.atlassian.jira.cluster.distribution.localq.rmi.CachingRMICachePeerManager.withCachePeer(CachingRMICachePeerManager.java:94) at com.atlassian.jira.cluster.distribution.localq.rmi.LocalQCacheOpRMISender.send(LocalQCacheOpRMISender.java:65) ... 6 more
Environment
Jira Data Center 9.X
.
Steps to Reproduce
- Create a user from a third-party plugin code that runs for example in a Caesium thread , Jira will run getClass().getClassloader()and use it in the RMI call , which will be failing for not finding that class
Expected Results
Jira uses its own classloader and cache replication finishes correctly
Actual Results
Jira fails in a race condition and uses the wrong classloader , causing the RMI invocation to fail at the destination node
Workaround
A rolling restart of all Jira's nodes
Attachments
Issue Links
- relates to
-
JRASERVER-75889 User cache replication sometimes fails due to the class loader being overridden
- Gathering Impact
- is related to
-
RUM-1645 Loading...
- mentioned in
-
Page Loading...