[CWD-1505] User/Group/Membership Import fails when using MySQL

Type: Bug
Resolution: Fixed
Priority: Medium
Fix Version/s: 2.0, Sprint 1
Affects Version/s: 1.6, 1.6.1
Component/s: None
Labels:
None

Bug Fix Policy:
View Atlassian Server bug fix policy

The Import Users feature does not perform a complete import when using MySQL.

Versions used during test:
MySQL 5.0.37
MySQL 5.0.67
mysql-connector-java-3.1.14
mysql-connector-java-5.0.8
mysql-connector-java-5.1.6

Problem reproduced when using the Atlassian Importer (with Confluence) and the CSV Importer.

The following errors are displayed:

2009-04-29 15:52:49,212 http-8095-Processor23 ERROR [crowd.importer.importers.BaseImporter] Could not add memberships for group 'confluence-users' as it does not exist in directory with ID: 360450
com.atlassian.crowd.integration.exception.ObjectNotFoundException: Failed to find com.atlassian.crowd.integration.model.RemoteGroup DirectoryID: 360450 and name: confluence-users
	at com.atlassian.crowd.model.group.GroupDAOHibernate.findByName(GroupDAOHibernate.java:85)
	at com.atlassian.crowd.integration.directory.internal.InternalDirectory.findGroupByName(InternalDirectory.java:374)
	at com.atlassian.crowd.integration.directory.internal.InternalDirectory.findGroupByName(InternalDirectory.java:357)
	at com.atlassian.crowd.manager.directory.bulk.BulkAddGroupMembershipsOperation.performFindContainerByName(BulkAddGroupMembershipsOperation.java:27)
	at com.atlassian.crowd.manager.directory.bulk.BulkAddGroupMembershipsOperation.performFindContainerByName(BulkAddGroupMembershipsOperation.java:13)
	at com.atlassian.crowd.manager.directory.DirectoryManagerGeneric.addAllPrincipalsToContainer(DirectoryManagerGeneric.java:298)
	at com.atlassian.crowd.manager.directory.DirectoryManagerGeneric.addAllPrincipalsToGroup(DirectoryManagerGeneric.java:344)



2009-04-03 14:25:33,751 http-8095-Processor18 ERROR [persistence.hibernate.batch.BatchProcessorImpl] Could not process class com.atlassian.crowd.integration.model.RemotePrincipal: xxxxx@gmail.com
org.hibernate.PropertyValueException: not-null property references a null or transient value: com.atlassian.crowd.integration.model.RemotePrincipal.credentials.credential
	at org.hibernate.engine.Nullability.checkNullability(Nullability.java:84)
	at org.hibernate.event.def.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:290)
	at org.hibernate.event.def.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:181)
	at org.hibernate.event.def.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:121)
	at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:187)
	at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:172)
	at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:94)
	at org.hibernate.event.def.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:70)
	at org.hibernate.impl.SessionImpl.fireSaveOrUpdate(SessionImpl.java:507)
	at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:499)
	at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:495)
	at com.atlassian.crowd.util.persistence.hibernate.batch.operation.SaveOrUpdateOperation.performOperation(SaveOrUpdateOperation.java:10)
	at com.atlassian.crowd.util.persistence.hibernate.batch.BatchProcessorImpl.processIndividual(BatchProcessorImpl.java:302)
	at com.atlassian.crowd.util.persistence.hibernate.batch.BatchProcessorImpl.processBatch(BatchProcessorImpl.java:279)



2009-04-03 14:25:33,749 http-8095-Processor18 ERROR [persistence.hibernate.batch.BatchProcessorImpl] Could not process class com.atlassian.crowd.integration.model.RemotePrincipal: cccc@yahoo.com
org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
	at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:61)
	at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:46)
	at org.hibernate.jdbc.BatchingBatcher.checkRowCounts(BatchingBatcher.java:68)
	at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:48)
	at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:246)
	at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:92)
	at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:87)
	at org.hibernate.jdbc.AbstractBatcher.prepareBatchStatement(AbstractBatcher.java:222)
	at org.hibernate.persister.collection.AbstractCollectionPersister.recreate(AbstractCollectionPersister.java:1119)
	at org.hibernate.action.CollectionRecreateAction.execute(CollectionRecreateAction.java:26)
	at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:250)
	at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:234)

Note:
There is a known bug with MySQL JDBC driver version 5.1.23 on 'RewriteBatch' error:

https://bugs.mysql.com/bug.php?id=68562
Using MySQL JDBC driver 5.1.48 resolved the issue

is duplicated by

CWD-1739 import users does not respect group memberships on first attempt

Closed

jmo added a comment - 24/Oct/2017 5:06 PM

I just ran into this with Crowd 3.0.1 and MySQL 5.6. I am not able to change transaction-isolation on this server so I just ran the Atlassian Importer (from a JIRA db) twice, and the second run appears to have attached all the correct groups to the users who were imported on the first run.

Hope there isn't other data corruption on import that I am not seeing...

jmo added a comment - 24/Oct/2017 5:06 PM I just ran into this with Crowd 3.0.1 and MySQL 5.6. I am not able to change transaction-isolation on this server so I just ran the Atlassian Importer (from a JIRA db) twice, and the second run appears to have attached all the correct groups to the users who were imported on the first run. Hope there isn't other data corruption on import that I am not seeing...

prdonahue added a comment - 24/Feb/2014 9:13 PM

Telling users to make wholesale changes to their server-wide MySQL configuration is a half-assed solution. And it ignores the reality that most people host more than just Atlassian apps on their MySQL instances. A more responsible solution would be to indicate how READ-COMMITTED can be set at the session level, not the global level.

prdonahue added a comment - 24/Feb/2014 9:13 PM Telling users to make wholesale changes to their server-wide MySQL configuration is a half-assed solution. And it ignores the reality that most people host more than just Atlassian apps on their MySQL instances. A more responsible solution would be to indicate how READ-COMMITTED can be set at the session level, not the global level.

SarahA added a comment - 02/Jun/2009 1:21 AM

We have updated the Crowd documentation:

The MySQL configuration guide: http://confluence.atlassian.com/display/CROWD/MySQL
A new FAQ: http://confluence.atlassian.com/display/CROWD/Problems+when+Importing+Users+into+MySQL

SarahA added a comment - 02/Jun/2009 1:21 AM We have updated the Crowd documentation: The MySQL configuration guide: http://confluence.atlassian.com/display/CROWD/MySQL A new FAQ: http://confluence.atlassian.com/display/CROWD/Problems+when+Importing+Users+into+MySQL

Peggy added a comment - 18/May/2009 12:44 AM - edited

I have also been able to duplicate the issue where when importing users/groups the memberships are not imported. The problem was identified that mySQL by default uses 'Repeatable Read' which can result in inconsistent reads and therefore wrong behaviour when adding memberships. The solution is to change from 'Repeatable Read' to 'Read Committed"

To change this setting, modify my.cnf so it includes transaction-isolation = READ-COMMITTED
eg:

[mysqld]
transaction-isolation = READ-COMMITTED

For more information on:

Repeatable Read, Read Committed see: mySQL Transactions
modifying my.cnf see: mySQL Option files

Peggy added a comment - 18/May/2009 12:44 AM - edited I have also been able to duplicate the issue where when importing users/groups the memberships are not imported. The problem was identified that mySQL by default uses 'Repeatable Read' which can result in inconsistent reads and therefore wrong behaviour when adding memberships. The solution is to change from 'Repeatable Read' to 'Read Committed" To change this setting, modify my.cnf so it includes transaction-isolation = READ-COMMITTED eg: [mysqld] transaction-isolation = READ-COMMITTED For more information on: Repeatable Read, Read Committed see: mySQL Transactions modifying my.cnf see: mySQL Option files

Details

Description

Attachments

Issue Links

Forms

Activity

Collapse comment: jmo added a comment - 24/Oct/2017 5:06 PM

Expand comment: jmo added a comment - 24/Oct/2017 5:06 PM

Collapse comment: prdonahue added a comment - 24/Feb/2014 9:13 PM

Expand comment: prdonahue added a comment - 24/Feb/2014 9:13 PM

Collapse comment: SarahA added a comment - 02/Jun/2009 1:21 AM

Expand comment: SarahA added a comment - 02/Jun/2009 1:21 AM

Collapse comment: Peggy added a comment - 18/May/2009 12:44 AM, Edited by Peggy - 18/May/2009 12:50 AM

Expand comment: Peggy added a comment - 18/May/2009 12:44 AM, Edited by Peggy - 18/May/2009 12:50 AM

People

Dates

Time Tracking