-
Bug
-
Resolution: Fixed
-
Low
-
None
-
None
-
None
Problem
If the cookie.tokenkey is not set in crowd.properties, any Atlassian application integrating with it will perform additional calls to the remote Crowd instance to get it from the cookie configuration. In this scenario it's possible an XML deserialisation will happen in the Crowd client in a non-optimal way, resulting in WebappClassLoader lock congestion. This can cause an instance to become unresponsive ("crash").
Verification
Taking thread dumps during a problematic period and analysing them will show the below stack traces:
"catalina-exec-7615" daemon prio=10 tid=0x00002aaae5ba2000 nid=0xa4c waiting for monitor entry [0x0000000066d72000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java) - waiting to lock <0x00000005453089d8> (a org.apache.catalina.loader.WebappClassLoader) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1556) at org.apache.xerces.parsers.ObjectFactory.findProviderClass(Unknown Source) at org.apache.xerces.parsers.ObjectFactory.newInstance(Unknown Source) at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source) at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source) at org.apache.xerces.parsers.SAXParser.<init>(Unknown Source) at org.apache.xerces.parsers.SAXParser.<init>(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.<init>(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl.<init>(Unknown Source) at org.apache.xerces.jaxp.SAXParserFactoryImpl.newSAXParser(Unknown Source) at javax.xml.bind.helpers.AbstractUnmarshallerImpl.getXMLReader(AbstractUnmarshallerImpl.java:106) at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:235) at javax.xml.bind.JAXB.unmarshal(JAXB.java:223) at com.atlassian.crowd.integration.rest.service.RestExecutor$MethodExecutor.andReceive(RestExecutor.java:345) at com.atlassian.crowd.integration.rest.service.RestCrowdClient.getCookieConfiguration(RestCrowdClient.java:1017) at com.atlassian.crowd.integration.http.CrowdHttpAuthenticatorImpl.getCookieTokenKey(CrowdHttpAuthenticatorImpl.java:214) at com.atlassian.crowd.integration.http.CrowdHttpAuthenticatorImpl.getUser(CrowdHttpAuthenticatorImpl.java:48) at com.atlassian.crowd.integration.http.CacheAwareCrowdHttpAuthenticator.getUser(CacheAwareCrowdHttpAuthenticator.java:28) at com.atlassian.crowd.integration.seraph.v25.CrowdAuthenticator.getUser(CrowdAuthenticator.java:364) at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:132) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) ... at java.lang.Thread.run(Thread.java:724)
Note on fix
Problem contains 3 parts:
- Checking cookie.tokenkey value in crowd.properties file and triggering a remote call to the Crowd server when the value is not present
- creating JAXBContext classes - this is fixed
- in a newly created Unmarshaller creating new SAXParser (not fixed, creates most of contention)
Workaround
Set the cookie.tokenkey to the value configured in the Crowd server. This is detailed in The crowd.properties file. Specifically:
- Check the crowd.properties file in Crowd to see if the cookie.tokenkey has been specified. If it has not then it will be the default value of crowd.token_key.
- Change the crowd.properties file for every application that is using Crowd to have the value set to either the value from step 1 or the default of crowd.token_key.
The location of crowd.properties file is different for each application. See each link for more information:
An example of this configuration:
application.name theplanetarium application.password sauronlosthiscontact application.login.url https://jira.atlassian.com/ crowd.server.url http://crowd.atlassian.com:8095/crowd/services/ crowd.base.url http://crowd.atlassian.com:8095/crowd/ session.isauthenticated session.isauthenticated session.tokenkey session.tokenkey session.validationinterval 2 session.lastvalidation session.lastvalidation cookie.tokenkey crowd.token_key
- is caused by
-
CWD-5100 The default crowd.properties file for clients should include the cookie name
- Closed
- relates to
-
CWD-4196 Crowd http-client token validation can trigger a flood of requests
- Closed
-
JRASERVER-26830 JIRA is extremely slow when Nested group is enabled in LDAP and lone child-groups are imported due to improper search filter
- Closed
-
CWD-5250 Cache cookie.tokenkey pulled from Crowd's /rest/config/cookie
- Future Consideration
-
JDEV-29458 Loading...
-
KRAK-256 Loading...
- mentioned in
-
Page Loading...