Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-18659

Unable to insert emoji into USER_COMMIT. COMMIT_COMMENT_CLOB

XMLWordPrintable

      2017-08-16 15:23:43,435 INFO [10-BAM::PlanExec:pool-17-thread-1] [ChangeDetectionListenerAction] : Change detection found 2 changes for plan PROJ-PLAN
2017-08-16 15:23:43,582 INFO [10-BAM::PlanExec:pool-17-thread-1] [AbstractBatchImpl] HHH000010: On release of batch it still contained JDBC statements
2017-08-16 15:23:43,582 WARN [10-BAM::PlanExec:pool-17-thread-1] [SqlExceptionHelper] SQL Error: 1366, SQLState: HY000
2017-08-16 15:23:43,582 ERROR [10-BAM::PlanExec:pool-17-thread-1] [SqlExceptionHelper] Incorrect string value: '\xF0\x9F\x98\x80' for column 'COMMIT_COMMENT_CLOB' at row 1
2017-08-16 15:23:43,582 ERROR [10-BAM::PlanExec:pool-17-thread-1] [BatchingBatch] HHH000315: Exception executing batch [could not execute batch]
2017-08-16 15:23:43,583 ERROR [10-BAM::PlanExec:pool-17-thread-1] [SessionImpl] HHH000346: Error during managed flush [could not execute batch]
2017-08-16 15:23:43,600 ERROR [10-BAM::PlanExec:pool-17-thread-1] [ChainExecutionManagerImpl] Plan 'PROJ-PLAN' could not be started. Exception: JDBC exception on Hibernate data access: SQLException for SQL [insert into USER_COMMIT (REPOSITORY_CHANGESET_ID, AUTHOR_ID, COMMIT_DATE, COMMIT_REVISION, COMMIT_COMMENT_CLOB, FOREIGN_COMMIT, COMMIT_ID) values (?, ?, ?, ?, ?, ?, ?)]; SQL state [HY000]; error code [1366]; could not execute batch; nested exception is org.hibernate.exception.GenericJDBCException: could not execute batch
org.springframework.orm.hibernate5.HibernateJdbcException: JDBC exception on Hibernate data access: SQLException for SQL [insert into USER_COMMIT (REPOSITORY_CHANGESET_ID, AUTHOR_ID, COMMIT_DATE, COMMIT_REVISION, COMMIT_COMMENT_CLOB, FOREIGN_COMMIT, COMMIT_ID) values (?, ?, ?, ?, ?, ?, ?)]; SQL state [HY000]; error code [1366]; could not execute batch; nested exception is org.hibernate.exception.GenericJDBCException: could not execute batch
        at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:171)
        at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:739)
        at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:588)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
        at sun.reflect.GeneratedMethodAccessor171.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
        at com.sun.proxy.$Proxy64.commit(Unknown Source)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:484)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at com.atlassian.bamboo.author.AuthorCreatorServiceInterceptor.invoke(AuthorCreatorServiceInterceptor.java:40)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
        at com.sun.proxy.$Proxy219.create(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor1919.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
        at com.sun.proxy.$Proxy216.create(Unknown Source)
        at com.atlassian.bamboo.chains.ChainExecutionManagerImpl.createNewChainState(ChainExecutionManagerImpl.java:524)
        at com.atlassian.bamboo.chains.ChainExecutionManagerImpl.lambda$start$0(ChainExecutionManagerImpl.java:223)
        at com.atlassian.bamboo.chains.ChainExecutionManagerImpl.tryStartChainState(ChainExecutionManagerImpl.java:314)
        at com.atlassian.bamboo.chains.ChainExecutionManagerImpl.start(ChainExecutionManagerImpl.java:221)
        at sun.reflect.GeneratedMethodAccessor1918.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
	...
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9F\x98\x80' for column 'COMMIT_COMMENT_CLOB' at row 1
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
        at com.mysql.jdbc.Util.getInstance(Util.java:387)
        at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1154)
        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
        at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1257)
        at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959)
        at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544)
        at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:110)
        ... 94 more
Caused by: java.sql.SQLException: Incorrect string value: '\xF0\x9F\x98\x80' for column 'COMMIT_COMMENT_CLOB' at row 1
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:957)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
        at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073)
        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1751)
        ... 98 more
2017-08-16 15:23:43,601 INFO [10-BAM::PlanExec:pool-17-thread-1] [DefaultErrorHandler] Recording an error: Plan 'PROJ-PLAN' could not be started. Exception: JDBC exception on Hibernate data access: SQLException for SQL [insert into USER_COMMIT (REPOSITORY_CHANGESET_ID, AUTHOR_ID, COMMIT_DATE, COMMIT_REVISION, COMMIT_COMMENT_CLOB, FOREIGN_COMMIT, COMMIT_ID) values (?, ?, ?, ?, ?, ?, ?)]; SQL state [HY000]; error code [1366]; could not execute batch; nested exception is org.hibernate.exception.GenericJDBCException: could not execute batch : PROJ-PLAN : JDBC exception on Hibernate data access: SQLException for SQL [insert into USER_COMMIT (REPOSITORY_CHANGESET_ID, AUTHOR_ID, COMMIT_DATE, COMMIT_REVISION, COMMIT_COMMENT_CLOB, FOREIGN_COMMIT, COMMIT_ID) values (?, ?, ?, ?, ?, ?, ?)]; SQL state [HY000]; error code [1366]; could not execute batch; nested exception is org.hibernate.exception.GenericJDBCException: could not execute batch

            [BAM-18659] Unable to insert emoji into USER_COMMIT. COMMIT_COMMENT_CLOB

            SIMON added a comment - - edited

            Hi Darren Warner,

            This indeed fix the issue, where now, the build is working fine and I see the EMOJI in the comment (not that I care about the EMOJI).

            For the next person that would get that issue, please note that if you are using the MySQL driver of version 5.1.x you need at least the 5.1.47 or later in order for that fix to work.

            That said, there still a warning in the health status of Bamboo, but it is only a warning. Still annoying, but something I can live with; for now.

             

            P.S: This is post the issue: I had to do this as well for the deployment part of that problematic build.

            ALTER TABLE DEPLOYMENT_VERSION_COMMIT MODIFY COMMIT_COMMENT_CLOB LONGTEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;

            Hope this help someone.

            SIMON added a comment - - edited Hi Darren Warner, This indeed fix the issue, where now, the build is working fine and I see the EMOJI in the comment (not that I care about the EMOJI). For the next person that would get that issue, please note that if you are using the MySQL driver of version 5.1.x you need at least the 5.1.47 or later in order for that fix to work. That said, there still a warning in the health status of Bamboo, but it is only a warning. Still annoying, but something I can live with; for now.   P.S: This is post the issue: I had to do this as well for the deployment part of that problematic build. ALTER TABLE DEPLOYMENT_VERSION_COMMIT MODIFY COMMIT_COMMENT_CLOB LONGTEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin; Hope this help someone.

            Darren Warner added a comment -

            I've managed to workaround this by changing the character set and collation for the user commits comment only:

            ALTER TABLE user_commit MODIFY commit_comment_clob LONGTEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;

            The JDBC connection string in bamboo.cfg.xml also needs to support multi-byte strings:

            jdbc:mysql://[host]/[database]?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8

            (restart Bamboo after applying these changes)

            So far this has allowed builds to succeed and the intent is to support utfmb4 just enough to avoid the problems cited in BAM-18703, which seem to relate to branch names. YMMV.

             

            Darren Warner added a comment - I've managed to workaround this by changing the character set and collation for the user commits comment only: ALTER TABLE user_commit MODIFY commit_comment_clob LONGTEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin; The JDBC connection string in bamboo.cfg.xml also needs to support multi-byte strings: jdbc:mysql: //[host]/[database]?autoReconnect= true &useUnicode= true &characterEncoding=UTF-8 (restart Bamboo after applying these changes) So far this has allowed builds to succeed and the intent is to support utfmb4 just enough to avoid the problems cited in BAM-18703 , which seem to relate to branch names. YMMV.  

            Bill Carreon (Inactive) added a comment -

            This bug is now being tracked in BAM-18703.

            Bill Carreon (Inactive) added a comment - This bug is now being tracked in BAM-18703 .

            Colin Viebrock added a comment -

            Just commenting on this because I don't think the above statement is true.  MySQL 5.5.3 (released 2010) supports `utf8mb4` character set, and there are lots of posts online outlining how to convert your database to use that character set:

            As it is, a commit message containing an emoji character, and committed to our Bitbucket-hosted repo, is now causing our Bamboo builds to fail ... and there is no way to fix it, as far as I can see.

            Colin Viebrock added a comment - Just commenting on this because I don't think the above statement is true.  MySQL 5.5.3 (released 2010) supports `utf8mb4` character set, and there are lots of posts online outlining how to convert your database to use that character set: https://dev.mysql.com/doc/relnotes/mysql/5.5/en/news-5-5-3.html https://mathiasbynens.be/notes/mysql-utf8mb4 https://gist.github.com/samstoller/1f7b3d54eb8c2bee47ab etc. As it is, a commit message containing an emoji character, and committed to our Bitbucket-hosted repo, is now causing our Bamboo builds to fail ... and there is no way to fix it, as far as I can see.

            Marcin Gardias added a comment -

            This is DBMS limitation: utf8 encoding in MySQL doesn't support 4-byte characters (in other words: it's not a correct utf8 implementation)

            Marcin Gardias added a comment - This is DBMS limitation: utf8 encoding in MySQL doesn't support 4-byte characters (in other words: it's not a correct utf8 implementation)

              Unassigned Unassigned
              kbrazulewicz Krystian Brazulewicz
              Affected customers:
              0 This affects my team
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: