Issue Details (XML | Word | Printable)

Key: CONF-12201
Type: Bug Bug
Status: Open Open
Priority: Critical Critical
Assignee: Unassigned
Reporter: Igor Minar
Votes: 1
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Confluence

Race condition issue with DefaultHibernateUser

Created: 20/Jun/08 12:56 AM   Updated: 03/Jul/08 01:54 AM
Component/s: Database / Hibernate, Users & Groups
Affects Version/s: 2.8
Fix Version/s: None

Time Tracking:
Not Specified

Participants: Anatoli Kazatchkov [Atlassian] and Igor Minar
Since last comment: 16 weeks ago
Internal Complexity: 4
Internal Value: 4
Labels:


 Description  « Hide
While debugging an issue in our custom authenticator that creates and updates user accounts during logins, I found something that appears to be a bug in how DefaultHibernateUser objects are being handled (and cached?) in confluence.

It looks like the DefaultHibernateUser objects are being cached with its (groups) collections attached with the hibernate session that performed an operation on the object recently (in a different thread). When another thread with a new hibernate session attempts to update such a user instance the exception below is thrown. We are not passing the objects between threads via http session or in any other way. Each thread obtains an instance of the user class via userAccessor.

Steps to reproduce:

You need two threads that get the user instance via an instance of UserAccessor, and call UserAccessor#saveUser(confUser).

  1. thread 1 opens a hibernate session
  2. thread 2 opens a hibernate session
  3. thread 1 fetches a user
  4. thread 1 saves the user
  5. thread 2 fetches the same user as thread #1
  6. thread 2 saves the user
  7. the exception is thrown in thread 2
  8. thread 1 closes the hibernate session - this is important, it seems that if the session was closed before thread 2 called saveUser, everything would be ok
  9. thread 2 closes the hibernate session

The exception:

org.springframework.orm.hibernate.HibernateSystemException: Illegal attempt to associate a collection with two open sessions; 
nested exception is net.sf.hibernate.HibernateException: Illegal attempt to associate a collection with two open sessions
Caused by: net.sf.hibernate.HibernateException: Illegal attempt to associate a collection with two open sessions
        at net.sf.hibernate.collection.PersistentCollection.setCurrentSession(PersistentCollection.java:257)
        at net.sf.hibernate.impl.OnUpdateVisitor.processCollection(OnUpdateVisitor.java:38)
        at net.sf.hibernate.impl.AbstractVisitor.processValue(AbstractVisitor.java:69)
        at net.sf.hibernate.impl.AbstractVisitor.processValues(AbstractVisitor.java:36)
        at net.sf.hibernate.impl.AbstractVisitor.process(AbstractVisitor.java:91)
        at net.sf.hibernate.impl.SessionImpl.doUpdateMutable(SessionImpl.java:1466)
        at net.sf.hibernate.impl.SessionImpl.doUpdate(SessionImpl.java:1480)
        at net.sf.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:1403)
        at org.springframework.orm.hibernate.HibernateTemplate$12.doInHibernate(HibernateTemplate.java:604)
        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)
        at org.springframework.orm.hibernate.HibernateTemplate.saveOrUpdate(HibernateTemplate.java:601)
        at com.atlassian.user.impl.hibernate.HibernateUserManager.saveUser(HibernateUserManager.java:171)
        at com.atlassian.user.impl.cache.CachingUserManager.saveUser(CachingUserManager.java:143)
        at com.atlassian.user.impl.delegation.DelegatingUserManager.saveUser(DelegatingUserManager.java:144)
        at bucket.user.DefaultUserAccessor.saveUser(DefaultUserAccessor.java:224)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:296)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:177)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
        at com.atlassian.spring.interceptors.SpringProfilingInterceptor.invoke(SpringProfilingInterceptor.java:20)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy22.saveUser(Unknown Source)
        at com.sun.dse.wikis.auth.ConfluenceUserManager$1.doInHibernate(ConfluenceUserManager.java:96)
        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)
        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:337)
        at com.sun.dse.wikis.auth.ConfluenceUserManager.updateUser(ConfluenceUserManager.java:79)
        at com.sun.dse.wikis.auth.SsoFilter.doFilter(SsoFilter.java:124)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:217)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:185)
        at com.atlassian.confluence.util.ClusterHeaderFilter.doFilter(ClusterHeaderFilter.java:35)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:217)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:185)
        at org.springframework.orm.hibernate.support.OpenSessionInV


 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Igor Minar added a comment - 20/Jun/08 03:31 PM - edited
Disconnecting the user object from the current session once I'm done using it seems to help.
user = userAccessor.getUser(name);
// do stuff with user
userAccessor.saveUser(user);
//commit transaction
session.evict(user)

But there is one more problem. If two threads (with different hibernate sessions) obtain two instances of the same user (when the cache is empty) and try to save them an exception is thrown:

org.springframework.orm.hibernate.HibernateSystemException: a different object with the same identifier value was already associated with the session: 13177238, 
of class: com.atlassian.user.impl.hibernate.DefaultHibernateUser; 
nested exception is net.sf.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session: 13177238, 
of class: com.atlassian.user.impl.hibernate.DefaultHibernateUser
Caused by: net.sf.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session: 13177238, 
of class: com.atlassian.user.impl.hibernate.DefaultHibernateUser
        at net.sf.hibernate.impl.SessionImpl.checkUniqueness(SessionImpl.java:1687)
        at net.sf.hibernate.impl.SessionImpl.doUpdateMutable(SessionImpl.java:1453)
        at net.sf.hibernate.impl.SessionImpl.doUpdate(SessionImpl.java:1480)
        at net.sf.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:1403)
        at org.springframework.orm.hibernate.HibernateTemplate$12.doInHibernate(HibernateTemplate.java:604)
        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)
        at org.springframework.orm.hibernate.HibernateTemplate.saveOrUpdate(HibernateTemplate.java:601)
        at com.atlassian.user.impl.hibernate.HibernateUserManager.saveUser(HibernateUserManager.java:171)
        at com.atlassian.user.impl.cache.CachingUserManager.saveUser(CachingUserManager.java:143)
        at com.atlassian.user.impl.delegation.DelegatingUserManager.saveUser(DelegatingUserManager.java:144)
        at bucket.user.DefaultUserAccessor.saveUser(DefaultUserAccessor.java:224)
        at sun.reflect.GeneratedMethodAccessor886.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:296)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:177)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
        at com.atlassian.spring.interceptors.SpringProfilingInterceptor.invoke(SpringProfilingInterceptor.java:20)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy22.saveUser(Unknown Source)
        at com.sun.dse.wikis.auth.ConfluenceUserManager$1.doInHibernate(ConfluenceUserManager.java:96)
        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)
        at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:337)
        at com.sun.dse.wikis.auth.ConfluenceUserManager.updateUser(ConfluenceUserManager.java:79)
        at com.sun.dse.wikis.auth.SsoFilter.doFilter(SsoFilter.java:124)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:217)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:185)
        at com.atlassian.confluence.util.ClusterHeaderFilter.doFilter(ClusterHeaderFilter.java:35)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:217)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:185)
        at com.atlassian.johnson.filters.AbstractJohnsonFilter.doFilter(AbstractJohnsonFilter.java:72)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:217)
        at org.apache.catalina.core.Appli

To me it looks like UserAccessor#getUser(String name) should be synchronized on the name object in order to avoid this.


Anatoli Kazatchkov [Atlassian] added a comment - 22/Jun/08 07:14 PM
Hi Igor,

This issues seems to look very much like your second problem. Can you please check what version of atlassian-user library you are using as the bug was fixed in 1.28. If you use the updated version of the library and still have the exception thrown can you please give more details on what you are trying to do when the exception is thrown (code snippet? )

If two threads (with different hibernate sessions) obtain two instances of the same user (when the cache is empty) and try to save them an exception is thrown:

I don't think this is the reason for the exception, after all it reports the same identifier withing one session.

Caused by: net.sf.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session: 13177238, 

The first reported problem is a bug. It happens because we use the collection from the cache that is already associated with an open hibernate session and try to attach it to a new session.
http://jira.atlassian.com/browse/USER-228


Igor Minar added a comment - 22/Jun/08 09:59 PM
Hi Anatoli,

Yes, the second issue looks like USER-239. We are using Confluence 2.8, which shipped with atlassian user 1.27. I'll upgrade to the latest version and check if we still see the issue.

The first problem is very likely caused by USER-228. Caching persistent objects is really a bad idea, especially if they are still attached to a session that retrieved them from the database. We have hundreds of exceptions in our logs because of this issue. It would be great if it got fixed soon. For now the workaround that I described earlier (using session.evict()) works, but that's pretty ugly.

Thanks for looking into this!

/i