[CWD-5806] Performing an LDAP sync logs ERROR: invalid byte sequence for encoding "UTF8": 0x00 error when using Postgres database

Type: Bug
Resolution: Fixed
Priority: Low
Fix Version/s: 5.1.0, 4.2.6, 4.3.9, 4.4.3, 5.0.2
Affects Version/s: 4.2.4, 4.4.0, 5.0.0
Component/s: Directory - LDAP
Labels:
None

Support reference count:
3
Symptom Severity:
Severity 3 - Minor
Bug Fix Policy:
View Atlassian Server bug fix policy

Issue Summary

Crowd (or embedded Crowd) is integrated with LDAP (e.g. Active Directory) in which LDAP throws back an exception (due to some error on LDAP side). Performing the LDAP sync, the atlassian-crowd.log shows this error:

Log snippet

2022-05-28 13:19:32,728 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] FULL synchronisation for directory [ 557057 ] starting
2022-05-28 13:19:32,887 Caesium-2-2 ERROR [jdbc.batch.internal.BatchingBatch] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	''DC=office,DC=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,DC=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,DC=mycompany,DC=com'
]; remaining name 'OU=people,DC=office,DC=mycompany,DC=com,DC=office,DC=mycompany,DC=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021  Call getNextException to see other errors in the batch.], SQL: update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?
2022-05-28 13:19:32,888 Caesium-2-2 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 0, SQLState: 22021
2022-05-28 13:19:32,888 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	''DC=office,DC=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,DC=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,DC=mycompany,DC=com'
]; remaining name 'OU=people,DC=office,DC=mycompany,DC=com,DC=office,DC=mycompany,DC=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021  Call getNextException to see other errors in the batch.
2022-05-28 13:19:32,889 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
2022-05-28 13:19:32,891 Caesium-2-2 ERROR [org.hibernate.internal.ExceptionMapperStandardImpl] HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute batch]
2022-05-28 13:19:32,893 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 557057 ] in [ 192ms ]

Steps to Reproduce

Install Crowd with Postgres database
Integrate Crowd with LDAP (e.g. tested with Active Directory)
Configure the Additional User DN and/or Additional Group DN that (incorrectly) includes the Base DN to purposely have an LDAP exception returned from AD, e.g.
- Base DN: dc=mycompany,dc=com
- Additional User DN: ou=users,dc=mycompany,dc=com
- Additional Group DN: ou=groups,dc=mycompany,dc=com

Expected Results

Crowd (and embedded Crowd) should be able to persist the LDAP exception into cwd_synchronisation_status and log into atlassian-crowd.log without throwing a "ERROR: invalid byte sequence for encoding "UTF8": 0x00" exception.

Actual Results

The below exception is thrown in the atlassian-crowd.log file:

2022-05-28 13:19:32,728 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] FULL synchronisation for directory [ 557057 ] starting
2022-05-28 13:19:32,887 Caesium-2-2 ERROR [jdbc.batch.internal.BatchingBatch] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021  Call getNextException to see other errors in the batch.], SQL: update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?
2022-05-28 13:19:32,888 Caesium-2-2 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 0, SQLState: 22021
2022-05-28 13:19:32,888 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021  Call getNextException to see other errors in the batch.
2022-05-28 13:19:32,889 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
2022-05-28 13:19:32,891 Caesium-2-2 ERROR [org.hibernate.internal.ExceptionMapperStandardImpl] HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute batch]
2022-05-28 13:19:32,893 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 557057 ] in [ 192ms ]
2022-05-28 13:19:32,998 Caesium-2-2 ERROR [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 557057 ].
org.springframework.dao.DataIntegrityViolationException: could not execute batch; SQL [update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?]; nested exception is org.hibernate.exception.DataException: could not execute batch
	at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:252)
	at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:802)
	at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:638)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:305)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy105.syncFailure(Unknown Source)
	at com.atlassian.crowd.manager.directory.SynchronisationStatusManagerImpl.syncFailure(SynchronisationStatusManagerImpl.java:94)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)
	at com.sun.proxy.$Proxy106.syncFailure(Unknown Source)
	at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1097)
	at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.lambda$synchronise$0(DirectorySynchroniserImpl.java:82)
	at com.atlassian.crowd.audit.AuditLogContextInternalImpl.withAuditLogSource(AuditLogContextInternalImpl.java:52)
	at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:80)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy112.synchronise(Unknown Source)
	at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:45)
	at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:92)
	at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
	at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)
	at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)
	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:435)
	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:430)
	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:454)
	at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:382)
	at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)
	at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)
	at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.exception.DataException: could not execute batch
	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:118)
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:129)
	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:105)
	at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:212)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:633)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:478)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:356)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1457)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:514)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3321)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2492)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:473)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:178)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:39)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:271)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
	at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:626)
	... 45 more
Caused by: java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
	'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021  Call getNextException to see other errors in the batch.
	at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:148)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2191)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:472)
	at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:791)
	at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1563)
	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544)
	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:119)
	... 62 more
Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2477)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2190)
	... 67 more
2022-05-28 13:19:34,156 http-nio-6424-exec-17 INFO [server.impl.application.WebApplicationImpl] Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'

Whilst atlassian-crowd.log is still able to log out the LDAP exception, downstream applications that use an affected embedded Crowd such as Confluence 7.11 throws an ERROR: invalid byte sequence for encoding "UTF8": 0x00 exception and stops the LDAP exception from appearing anywhere in Confluence database or Confluence logs.

See CONFSERVER-78869 for impact on Confluence

Workaround

N/A

depended on by

CONFSERVER-78869 Performing an LDAP sync results with ERROR: invalid byte sequence for encoding "UTF8": 0x00 error with Postgres database

Gathering Impact

is cloned by: KRAK-4708 Failed to load

mentioned in: Page Failed to load; Page Failed to load; Page Failed to load

Daniel Serkowski made changes - 23/Nov/2022 9:55 AM

Remote Link

Original: This issue links to "Page (Confluence)" [ 707361 ]

Daniel Serkowski made changes - 18/Nov/2022 6:00 PM

Remote Link

New: This issue links to "Page (Confluence)" [ 707686 ]

Daniel Serkowski made changes - 17/Nov/2022 5:23 PM

Remote Link

New: This issue links to "Page (Confluence)" [ 707361 ]

aanastasov made changes - 26/Jul/2022 10:20 AM

Resolution		New: Fixed [ 1 ]
Status	Original: Waiting for Release [ 12075 ]	New: Closed [ 6 ]

SET Analytics Bot made changes - 20/Jul/2022 4:03 AM

Support reference count

Original: 2

New: 3

Eric Lam made changes - 20/Jul/2022 3:51 AM

Description

Original: h3. Issue Summary

Crowd (or _embedded Crowd_) is integrated with LDAP (e.g. Active Directory) in which LDAP throws back an exception (due to some error on LDAP side). Performing the LDAP sync, the {{atlassian-crowd.log}} shows this error:

{code:title=Log snippet}
2022-05-28 13:19:32,728 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] FULL synchronisation for directory [ 557057 ] starting
2022-05-28 13:19:32,887 Caesium-2-2 ERROR [jdbc.batch.internal.BatchingBatch] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,DC=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; remaining name 'OU=people,DC=office,DC=mycompany,DC=com,DC=office,DC=mycompany,DC=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.], SQL: update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?
2022-05-28 13:19:32,888 Caesium-2-2 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 0, SQLState: 22021
2022-05-28 13:19:32,888 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,DC=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; remaining name 'OU=people,DC=office,DC=mycompany,DC=com,DC=office,DC=mycompany,DC=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.
2022-05-28 13:19:32,889 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
2022-05-28 13:19:32,891 Caesium-2-2 ERROR [org.hibernate.internal.ExceptionMapperStandardImpl] HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute batch]
2022-05-28 13:19:32,893 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 557057 ] in [ 192ms ]
{code}

h3. Steps to Reproduce

# Install Crowd
# Integrate Crowd with LDAP (e.g. tested with Active Directory)
# Configure the Additional User DN and/or Additional Group DN that (incorrectly) includes the Base DN to purposely have an LDAP exception returned from AD, e.g.
#* *Base DN*: dc=mycompany,dc=com
#* *Additional User DN*: ou=users,dc=mycompany,dc=com
#* *Additional Group DN*: ou=groups,dc=mycompany,dc=com

h3. Expected Results

Crowd (and _embedded Crowd_) should be able to persist the LDAP exception into {{cwd_synchronisation_status}} and log into {{atlassian-crowd.log}} without throwing a "ERROR: invalid byte sequence for encoding "UTF8": 0x00" exception.

h3. Actual Results

The below exception is thrown in the {{atlassian-crowd.log}} file:

{code}
2022-05-28 13:19:32,728 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] FULL synchronisation for directory [ 557057 ] starting
2022-05-28 13:19:32,887 Caesium-2-2 ERROR [jdbc.batch.internal.BatchingBatch] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.], SQL: update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?
2022-05-28 13:19:32,888 Caesium-2-2 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 0, SQLState: 22021
2022-05-28 13:19:32,888 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.
2022-05-28 13:19:32,889 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
2022-05-28 13:19:32,891 Caesium-2-2 ERROR [org.hibernate.internal.ExceptionMapperStandardImpl] HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute batch]
2022-05-28 13:19:32,893 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 557057 ] in [ 192ms ]
2022-05-28 13:19:32,998 Caesium-2-2 ERROR [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 557057 ].
org.springframework.dao.DataIntegrityViolationException: could not execute batch; SQL [update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?]; nested exception is org.hibernate.exception.DataException: could not execute batch
at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:252)
at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:802)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:638)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:305)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy105.syncFailure(Unknown Source)
at com.atlassian.crowd.manager.directory.SynchronisationStatusManagerImpl.syncFailure(SynchronisationStatusManagerImpl.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)
at com.sun.proxy.$Proxy106.syncFailure(Unknown Source)
at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1097)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.lambda$synchronise$0(DirectorySynchroniserImpl.java:82)
at com.atlassian.crowd.audit.AuditLogContextInternalImpl.withAuditLogSource(AuditLogContextInternalImpl.java:52)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy112.synchronise(Unknown Source)
at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:45)
at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:92)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:435)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:430)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:454)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:382)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.exception.DataException: could not execute batch
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:118)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:129)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:105)
at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:212)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:633)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:478)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:356)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1457)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:514)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3321)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2492)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:473)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:178)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:39)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:271)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:626)
... 45 more
Caused by: java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.
at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:148)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2191)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:472)
at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:791)
at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1563)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:119)
... 62 more
Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2477)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2190)
... 67 more
2022-05-28 13:19:34,156 http-nio-6424-exec-17 INFO [server.impl.application.WebApplicationImpl] Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'
{code}

Whilst {{atlassian-crowd.log}} is still able to log out the LDAP exception, downstream applications that use an affected _embedded Crowd_ such as Confluence 7.11 throws an *{{ERROR: invalid byte sequence for encoding "UTF8": 0x00}}* exception and stops the LDAP exception from appearing anywhere in Confluence database or Confluence logs.

* See [CONFSERVER-78869|https://jira.atlassian.com/browse/CONFSERVER-78869] for impact on Confluence

h3. Workaround

N/A

New: h3. Issue Summary

Crowd (or _embedded Crowd_) is integrated with LDAP (e.g. Active Directory) in which LDAP throws back an exception (due to some error on LDAP side). Performing the LDAP sync, the {{atlassian-crowd.log}} shows this error:

{code:title=Log snippet}
2022-05-28 13:19:32,728 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] FULL synchronisation for directory [ 557057 ] starting
2022-05-28 13:19:32,887 Caesium-2-2 ERROR [jdbc.batch.internal.BatchingBatch] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,DC=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; remaining name 'OU=people,DC=office,DC=mycompany,DC=com,DC=office,DC=mycompany,DC=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.], SQL: update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?
2022-05-28 13:19:32,888 Caesium-2-2 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 0, SQLState: 22021
2022-05-28 13:19:32,888 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,DC=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,DC=mycompany,DC=com'
]; remaining name 'OU=people,DC=office,DC=mycompany,DC=com,DC=office,DC=mycompany,DC=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.
2022-05-28 13:19:32,889 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
2022-05-28 13:19:32,891 Caesium-2-2 ERROR [org.hibernate.internal.ExceptionMapperStandardImpl] HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute batch]
2022-05-28 13:19:32,893 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 557057 ] in [ 192ms ]
{code}

h3. Steps to Reproduce

# Install Crowd *with Postgres database*
# Integrate Crowd with LDAP (e.g. tested with Active Directory)
# Configure the Additional User DN and/or Additional Group DN that (incorrectly) includes the Base DN to purposely have an LDAP exception returned from AD, e.g.
#* *Base DN*: dc=mycompany,dc=com
#* *Additional User DN*: ou=users,dc=mycompany,dc=com
#* *Additional Group DN*: ou=groups,dc=mycompany,dc=com

h3. Expected Results

Crowd (and _embedded Crowd_) should be able to persist the LDAP exception into {{cwd_synchronisation_status}} and log into {{atlassian-crowd.log}} without throwing a "ERROR: invalid byte sequence for encoding "UTF8": 0x00" exception.

h3. Actual Results

The below exception is thrown in the {{atlassian-crowd.log}} file:

{code}
2022-05-28 13:19:32,728 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] FULL synchronisation for directory [ 557057 ] starting
2022-05-28 13:19:32,887 Caesium-2-2 ERROR [jdbc.batch.internal.BatchingBatch] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.], SQL: update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?
2022-05-28 13:19:32,888 Caesium-2-2 WARN [engine.jdbc.spi.SqlExceptionHelper] SQL Error: 0, SQLState: 22021
2022-05-28 13:19:32,888 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.
2022-05-28 13:19:32,889 Caesium-2-2 ERROR [engine.jdbc.spi.SqlExceptionHelper] ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
2022-05-28 13:19:32,891 Caesium-2-2 ERROR [org.hibernate.internal.ExceptionMapperStandardImpl] HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute batch]
2022-05-28 13:19:32,893 Caesium-2-2 INFO [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 557057 ] in [ 192ms ]
2022-05-28 13:19:32,998 Caesium-2-2 ERROR [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 557057 ].
org.springframework.dao.DataIntegrityViolationException: could not execute batch; SQL [update cwd_synchronisation_status set directory_id=?, node_id=?, node_name=?, sync_start=?, sync_end=?, sync_status=?, status_parameters=?, incremental_sync_error=?, full_sync_error=? where id=?]; nested exception is org.hibernate.exception.DataException: could not execute batch
at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:252)
at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:802)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:638)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:305)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy105.syncFailure(Unknown Source)
at com.atlassian.crowd.manager.directory.SynchronisationStatusManagerImpl.syncFailure(SynchronisationStatusManagerImpl.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)
at com.sun.proxy.$Proxy106.syncFailure(Unknown Source)
at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1097)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.lambda$synchronise$0(DirectorySynchroniserImpl.java:82)
at com.atlassian.crowd.audit.AuditLogContextInternalImpl.withAuditLogSource(AuditLogContextInternalImpl.java:52)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy112.synchronise(Unknown Source)
at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:45)
at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:92)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:435)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:430)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:454)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:382)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.exception.DataException: could not execute batch
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:118)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:129)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:105)
at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:212)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:633)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:478)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:356)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1457)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:514)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3321)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2492)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:473)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:178)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:39)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:271)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:626)
... 45 more
Caused by: java.sql.BatchUpdateException: Batch entry 0 update cwd_synchronisation_status set directory_id=557057, node_id=NULL, node_name=NULL, sync_start=1653707972655, sync_end=NULL, sync_status='FULL', status_parameters='', incremental_sync_error=NULL, full_sync_error='OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
''DC=office,dc=mycompany,DC=com''
OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; nested exception is javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-0310023C, problem 2001 (NO_OBJECT), data 0, best match of:
'DC=office,dc=mycompany,DC=com'
]; remaining name 'OU=people,dc=office,dc=mycompany,dc=com,dc=office,dc=mycompany,dc=com'' where id=622593 was aborted: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021 Call getNextException to see other errors in the batch.
at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:148)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2191)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:472)
at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:791)
at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1563)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:119)
... 62 more
Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
  Location: File: wchar.c, Routine: report_invalid_encoding, Line: 2017
  Server SQLState: 22021
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2477)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2190)
... 67 more
2022-05-28 13:19:34,156 http-nio-6424-exec-17 INFO [server.impl.application.WebApplicationImpl] Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'
{code}

Whilst {{atlassian-crowd.log}} is still able to log out the LDAP exception, downstream applications that use an affected _embedded Crowd_ such as Confluence 7.11 throws an *{{ERROR: invalid byte sequence for encoding "UTF8": 0x00}}* exception and stops the LDAP exception from appearing anywhere in Confluence database or Confluence logs.

* See [CONFSERVER-78869|https://jira.atlassian.com/browse/CONFSERVER-78869] for impact on Confluence

h3. Workaround

N/A

Eric Lam made changes - 20/Jul/2022 3:51 AM

Summary

Original: Performing an LDAP sync logs ERROR: invalid byte sequence for encoding "UTF8": 0x00 error

New: Performing an LDAP sync logs ERROR: invalid byte sequence for encoding "UTF8": 0x00 error when using Postgres database

Daniel Serkowski made changes - 18/Jul/2022 1:03 PM

Fix Version/s		New: 5.1.0 [ 100694 ]
Fix Version/s		New: 5.0.2 [ 102201 ]
Fix Version/s		New: 4.4.3 [ 102200 ]
Fix Version/s		New: 4.3.9 [ 102199 ]
Fix Version/s		New: 4.2.6 [ 102198 ]

Daniel Serkowski made changes - 18/Jul/2022 1:01 PM

Status

Original: In Progress [ 3 ]

New: Waiting for Release [ 12075 ]

Rudy Slaiby made changes - 27/Jun/2022 10:06 AM

Remote Link

New: This issue links to "Page (Confluence)" [ 656856 ]

Assignee:: Daniel Serkowski

Reporter:: Eric Lam

Affected customers:: 0 This affects my team

Watchers:: 3 Start watching this issue

Created:: 28/May/2022 3:31 AM

Updated:: 23/Nov/2022 9:55 AM

Resolved:: 26/Jul/2022 10:20 AM

Details

Description

Issue Summary

Steps to Reproduce

Expected Results

Actual Results

Workaround

Attachments

Issue Links

Forms

Activity

People

Dates