Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-18509

MySQL databases incapable of handling 4byte UTF-8 Characters. Confluence should handle this gracefully

XMLWordPrintable

      NOTE: This suggestion is for Confluence Server. Using Confluence Cloud? See the corresponding suggestion.

      Atlassian Update - April 2020

      The Confluence server development team has now completed both item 1 (Support for emoji in MySQL) and item 2 (providing meaningful error messages). We strongly encourage customers to follow the steps outlined in our documentation to migrate from UTF8 to UTF8MB4, completely resolving this issue.

      If you are unable to migrate to UTF8MB4 we have added a number of meaningful error messages to Confluence to stop users attempting to add unsupported 4-byte characters. We have added these errors to:

      • page publish/save events for pages and blogs 
      • page titles
      • comment/inline comment saves.

      Please note that we have not added this error message to content fields that are less used/less likely to contain 4byte characters including:

      • comments accessed via email,
      • the editor in Questions for Confluence,
      • comments on attachments,
      • gadget comments,
      • attachment titles
      • content added via Confluence Mobile. 

      From the Confluence development team, we want to say thank you for everyone who provided feedback as part of both of the fixes mentioned above. We hope you now enjoy an emoji-filled Confluence. 

      Regards,

      The Confluence Server Team

      The following error:

      org.springframework.jdbc.UncategorizedSQLException: Hibernate operation: could not update: com.atlassian.confluence.core.BodyContent#38141954; uncategorized SQLException for SQL []; SQL state [HY000]; error code [1366]; Incorrect string value: '\xF4\x80\x82\x84 z...' for column 'BODY' at row 1; nested exception is java.sql.SQLException: Incorrect string value: '\xF4\x80\x82\x84 z...' for column 'BODY' at row 1
Caused by: java.sql.SQLException: Incorrect string value: '\xF4\x80\x82\x84 z...' for column 'BODY' at row 1
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3536)

      is thrown by confluence when your try to save a page with bullet points copied from an external source (Pdf,etc).

      It appears to be directly related to this MySQL limitation: http://bugs.mysql.com/bug.php?id=28853

      We should handle this gracefully since it is a known limitation for MySQL.

      Note: This errors can also happen in the following situations:

      1. Restoring an XML backup into a MySQL database
      2. Attempting to log into Confluence
        For the login failure, the error message will look like this: 
        url:/dologin.action] [engine.jdbc.spi.SqlExceptionHelper] logExceptions Incorrect string value: '\xF0\x9F\x98\x8E p...' for column 'description' at row 1
      Workaround for XML imports:

      Use XML cleaner from this KB.

      Also documented here

      Workaround for login issues

      If you are seeing the Incorrect string value errors for the /dologin.action requests, proceed as follows:

      1. Find the offending groups on the AD side and remove the 4-byte characters from the description based on the memberships of the user that cannot log in, or
      2. Change the Description attribute to the CN instead on the directory settings in Confluence

        1. image.png
          image.png
          61 kB

              ggautam Ganesh Gautam
              pkamal Partha
              Votes:
              185 Vote for this issue
              Watchers:
              176 Start watching this issue

                Created:
                Updated:
                Resolved: