Uploaded image for project: 'Crowd Data Center'
  1. Crowd Data Center
  2. CWD-5357

AzureAD sync errors out when duplicate groups are encountered

    • 29
    • 12
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Issue Obsolete
      Issue Summary

      Crowd can't sync when there are duplicate groups in AzureAD, as per mentioned in the following Documentation: Configuring Azure Active Directory - Atlassian Documentation

      In Azure AD, you can have multiple groups with the same name (displayName), but it's not supported in Crowd and results in a failing synchronization. Make sure you change your Azure AD group names to unique ones.

      Crowd needs to use the DN to distinguish between the groups that appear to be duplicated. Azure AD allows duplicate group names - Morgan Simonsen's Blog

      Active Directory is based on LDAP and in the LDAP naming scheme an object may have the same Relative Distinguished Name (RDN), as long as the Distinguished Name (DN) is unique. In this case the RDN is the leaf name and the DN is the fully qualified name. So you can have two users named John Doe as long as they do not both reside within the same Organizational Unit (OU), or location, in the directory.

      Environment

      AzureAD & Crowd 3.1.2

      Expected Results

      synchronization succeded

      Actual Results

      Failure to finish sync.
      The below exception is thrown in the atlassian-crowd.log file:

      2018-11-15 11:04:47,308 Caesium-2-1 ERROR [hibernate.batch.hibernate5.Hibernate5BatchProcessor] Could not process class com.atlassian.crowd.model.group.InternalGroup: com.atlassian.crowd.model.group.InternalGroup@53977e56[id=2050678,name=groupOne,type=GROUP,active=true,description=description,lowerName=groupone,createdDate=Thu Nov 15 11:04:46 UTC 2018,updatedDate=Thu Nov 15 11:04:46 UTC 2018,directoryId=557057,externalId=5c432810-6d7e-497e-af41-195301e20c59]
      javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute batch
      	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:147)
      	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:155)
      	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:162)
      	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1441)
      	at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1421)
      	at com.atlassian.crowd.util.persistence.hibernate.batch.hibernate5.Hibernate5BatchProcessor.flushSession(Hibernate5BatchProcessor.java:51)
      	at com.atlassian.crowd.util.persistence.hibernate.batch.hibernate5.AbstractHibernateBatchProcessor.commitTransaction(AbstractHibernateBatchProcessor.java:55)
      	at com.atlassian.crowd.util.persistence.hibernate.batch.hibernate5.AbstractHibernateBatchProcessor.afterProcessIndividual(AbstractHibernateBatchProcessor.java:22)
      	at com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor.processIndividual(AbstractBatchProcessor.java:173)
      	at com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor.processBatch(AbstractBatchProcessor.java:155)
      	at com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor.execute(AbstractBatchProcessor.java:125)
      	at com.atlassian.crowd.dao.group.GroupDAOHibernate.addAll(GroupDAOHibernate.java:77)
      	at com.atlassian.crowd.directory.CachingDirectory.addAllGroups(CachingDirectory.java:122)
      	at com.atlassian.crowd.directory.DbCachingRemoteChangeOperations.addGroups(DbCachingRemoteChangeOperations.java:482)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
      	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
      	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
      	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
      	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
      	at com.sun.proxy.$Proxy500.addGroups(Unknown Source)
      	at com.atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations.addOrUpdateCachedGroups(DirectoryCacheImplUsingChangeOperations.java:84)
      	at com.atlassian.crowd.directory.cache.DeltaQueryCacheRefresher.synchroniseAllGroups(DeltaQueryCacheRefresher.java:326)
      	at com.atlassian.crowd.directory.cache.DeltaQueryCacheRefresher.synchroniseAll(DeltaQueryCacheRefresher.java:91)
      	at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:968)
      	at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:71)
      	at sun.reflect.GeneratedMethodAccessor850.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
      	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
      	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
      	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
      	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
      	at com.sun.proxy.$Proxy83.synchronise(Unknown Source)
      	at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:45)
      	at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:85)
      	at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153)
      	at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118)
      	at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97)
      	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443)
      	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438)
      	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462)
      	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390)
      	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285)
      	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282)
      	at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65)
      	at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59)
      	at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: org.hibernate.exception.ConstraintViolationException: could not execute batch
      	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)
      	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
      	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
      	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:121)
      	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
      	at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
      	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:206)
      	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
      	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
      	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
      	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
      	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1435)
      	... 55 more
      Caused by: java.sql.BatchUpdateException: Batch entry 0 insert into cwd_group (group_name, lower_group_name, active, is_local, created_date, updated_date, description, group_type, directory_id, external_id, id) values ('groupOne', 'groupone', 'T', 'F', '2018-11-15 11:04:46.993000+00', '2018-11-15 11:04:46.993000+00', 'description', 'GROUP', 557057, '5c432810-6d7e-497e-af41-195301e20c59', 2050678) was aborted: ERROR: duplicate key value violates unique constraint "uk_group_name_dir_id"
        Detail: Key (lower_group_name, directory_id)=(groupone, 557057) already exists.
        Location: File: nbtinsert.c, Routine: _bt_check_unique, Line: 433
        Server SQLState: 23505  Call getNextException to see other errors in the batch.
      	at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:148)
      	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2190)
      	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:472)
      	at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:791)
      	at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1547)
      	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1135)
      	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:111)
      	... 63 more
      
      Workaround
      • Customers have to purchase additional Add-Ons/App to sync AzureAD, specifically Office 365 Directory Connector for Crowd (ODCC) Cleito, to get AzureAD to sync properly.
      • Ensure that there are no duplicated groups in Azure AD side
      • Find the duplicated groups from the error logs (From the above example, it's groupOne) and rename it to something unique
      Suggestion:

      Duplicated Azure AD groups should be imported with a suffix generated from another property of the AD object.

      Related Feature requests:
      • CWD-5019 Ability to Filter users and groups For AzureAD
      • CWD-5142 Allow Crowd to pull different Azure AD attributes

          Form Name

            [CWD-5357] AzureAD sync errors out when duplicate groups are encountered

            Winderson Souza made changes -
            Resolution New: Obsolete [ 11 ]
            Status Original: Gathering Interest [ 11772 ] New: Closed [ 6 ]

            From MS Document, creating duplicates should not be possible:

            It could've been in the past and I also found some bug that fixed this.

            Winderson Souza added a comment - From MS Document, creating duplicates should not be possible: https://learn.microsoft.com/bs-latn-ba/entra/fundamentals/how-to-manage-groups 5. Enter a Group name. Choose a name that you'll remember and that makes sense for the group. A check will be performed to determine if the name is already in use. If the name is already in use, you'll be asked to change the name of your group. It could've been in the past and I also found some bug that fixed this. https://github.com/MicrosoftDocs/azure-docs/issues/94121

            Luc added a comment -

            Agree with @aligntechsupport and @OllyMason.  We are also unable to use the connector because of this.

            Luc added a comment - Agree with @aligntechsupport and @OllyMason.  We are also unable to use the connector because of this.
            Tanya Gordon made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 524170 ]
            Dayana made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 521771 ]
            Tanya Gordon made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 500237 ]

            This issue is not obsolete.  We are still running into this problem on our Crowd instance (4.0.2) and Azure AD.  Azure AD still allows multiple groups with the same name and we have duplicate group names in our AD.  Our AD is fairly large and we are not able to prevent users from creating a duplicate group name in the directory so we need this resolved.

            Chris Beahm added a comment - This issue is not obsolete.  We are still running into this problem on our Crowd instance (4.0.2) and Azure AD.  Azure AD still allows multiple groups with the same name and we have duplicate group names in our AD.  Our AD is fairly large and we are not able to prevent users from creating a duplicate group name in the directory so we need this resolved.

            Seconding OllyMason. The issue still persists and prevents usage of the Azure AD connector.

            aligntechsupport added a comment - Seconding OllyMason. The issue still persists and prevents usage of the Azure AD connector.
            Dawid Owoc (Inactive) made changes -
            Affects Version/s Original: 3.1.2 [ 76994 ]
            Workflow Original: JAC Bug Workflow v3 [ 3365473 ] New: JAC Suggestion Workflow 3 [ 3813738 ]
            Issue Type Original: Bug [ 1 ] New: Suggestion [ 10000 ]
            Priority Original: Medium [ 3 ]
            Status Original: Needs Triage [ 10030 ] New: Gathering Interest [ 11772 ]
            SET Analytics Bot made changes -
            UIS Original: 44 New: 29

              Unassigned Unassigned
              wlintz Wade Lintz (Inactive)
              Votes:
              22 Vote for this issue
              Watchers:
              29 Start watching this issue

                Created:
                Updated:
                Resolved: