HipChat Data Center: Push & Email Notification Absence Due to License Whitespace

Summary

HipChat mobile users are not receiving push & email notifications in their devices according to the scenarios listed in the https://<fqdn>/account/notifications page

Environment

• HipChat Data Center 3.1.3

Steps to Reproduce

1. Log in to the mobile device
2. Leave it running in the background
3. Receive a @mention or 1-1 messages to trigger the push & email notification
4. Verify that the push notification is received

Expected Results

The mobile user should receive the push & email notification on their device accordingly

Actual Results

No push notification is delivered to the mobile device / no email notification is delivered to the inbox

Cause

We have identified that a Hipchat Data Center license generated shows up in our logs as having a problem being parsed by one of our external servers that controls push notifications (called barb).

It is confirmed that the issue is related to the license having line breaks at certain points and may cause problems with users receiving push notifications on their mobile devices using the Hipchat iOS and Android applications / email notification. If your users aren't receiving push / email notifications, then this may be the cause.

Workaround

The issue was identified as being related to line breaks. To fix this, we will need to remove these breaks and re-apply the license. This should not restart any services, however it's advised to do this during maintenance window if possible.

1. Copy this license from your account at my.atlassian.com and paste into a text editor.
2. The format will be in base64 block quote, which will look similar to this in my.atlassian.com (this is an example, your license will be different):

   AAABRg0ODAoPeNp1kF9LwzAUxd/zKQK+6EPHUu3mBgFrW7ZCu6mdgiDIXb1zgS4tSTrctzdtN6aFv
   d0/3HN/51ylpaR+pSgbUTaZuqOp59IgXFF3yO7JVlT5FsynRrVHNcgMKIOKb6DQSIJSGsjNAnbI4
   QukgOIBTAFaC5CDvNyRp1rZc40hGOSNoDO8c9iYJCJHqTH6qYQ6/Fl6zfIoG6Ugigu6PaxFvVujW
   m5eba+5w0jWzuOQP3oec2a3ceLE48nMmfvvzySr1zpXojKilNyoGvtyFggClI3Rdm1BbCNB5heIe
   /dHd6tDhW02wTJNo5cg9pOTcUuWxGEWLZyEMW809CZjYjv+f7JU39a8hhbUPwVAAoXtqB9qj8JGK
   PbYOSi6t282nkbLJSGeI5iLKrB39LrxTTvjNx9TGu2hqNtP5Fx2er8mU78vMC0CFQCKt+GqKi44j
   JoGILUcz9e8U5fE0AIUTHTnWC2r7NJ1l1VluqP8z0VMw7Y=X02g8
   

3. We will need to then remove each of the individual line breaks so the entire license is one string instead of 5 different lines. After completed, it'll look similar to this:

   XXXBRg0ODAoPeNp1kF9LwzAUxd/zKQK+6EPHUu3mBgFrW7ZCu6mdgiDIXb1zgS4tSTrctzdtN6aFvd0/3HN/51ylpaR+pSgbUTaZuqOp59IgXFF3yO7JVlT5FsynRrVHNcgMKIOKb6DQSIJSGsjNAnbI4QukgOIBTAFaC5CDvNyRp1rZc40hGOSNoDO8c9iYJCJHqTH6qYQ6/Fl6zfIoG6Ugigu6PaxFvVujWm5eba+5w0jWzuOQP3oec2a3ceLE48nMmfvvzySr1zpXojKilNyoGvtyFggClI3Rdm1BbCNB5heIe/dHd6tDhW02wTJNo5cg9pOTcUuWxGEWLZyEMW809CZjYjv+f7JU39a8hhbUPwVAAoXtqB9qj8JGKPbYOSi6t282nkbLJSGeI5iLKrB39LrxTTvjNx9TGu2hqNtP5Fx2er8mU78vMC0CFQCKt+GqKi44jJoGILUcz9e8U5fE0AIUTHTnWC2r7NJ1l1VluqP8z0VMw7Y=XXXX
   

4. Check to make sure there are no spaces when you remove the line breaks. Also verify that you have not deleted any characters as this is needed for license validity. Then copy the license as such and paste into the 'License' tab inside the HCDC cluster.
5. The details of the license should appear after applied.

Once done, proceed to test the push notification:

1. Log into the Hipchat web interface with your user credentials.
2. Navigate to 'My Account' on the left hand sidebar.
3. Next, select 'Sessions' link at the top right of the screen.
4. Verify that there are no clients connected. If there are, please log out of any client that shows connected.
5. Finally, log into your Hipchat mobile application. Once logged in, turn off your screen.
6. Have another user send you a 1-1 message or @mention you in a room that you're in.
7. This should trigger a notification on your device.
8. If it does not, please have the other user verify that your status is set to 'away' after you turn off the screen on the device and try again.
9. If it is and still not coming through, please log into the Hipchat server terminal / SSH console for each of the nodes and attach the file generated through the hipchat diagnostics -b command from each node to this support ticket together with the passphrase for review