Desktop/web clients show error 'Couldn't load HipChat' after making a change to FQDN

Summary

After updating the FQDN using the WebUI, the /etc/hosts file does not show the new FQDN. This results in the web/desktop apps to show an error:

"Couldn't load HipChat"

If you look at the solo.json file and the /etc/hosts after the change, it'll show two different FQDN's:

  ...
  "environment": "btf",
  "fqdn": "new-FQDN.test-server.com",
  "fqdn_public": "new-FQDN.test-server.com",
  ...
  "hostname": "new-FQDN",
  "ipaddress": "10.x.x.x",

admin@server-test:~$ cat /etc/hosts
127.0.0.1       old-FQDN.test-server.com old-hostname localhost localhost.localdom

Environment

HipChat Server 1.4.x, 2.0.x, 2.2.x
This error only appears to be related to AWS deployed instances.

Steps to Reproduce

1. Login to the HipChat Server web user interface
2. Change either the hostname/domain
3. Verify that this change resides in DNS as well
4. After DNS propagation, try and login with the web or desktop app.

Expected Results

The HipChat client should load.

Actual Results

Receive error 'Couldn't load HipChat'.

Notes

• This happens after a change to the FQDN is made (including on HC Server install steps)
• Amazon ELB address is also sometimes auto-populated instead of the HCS FQDN on new installs
• Only happen to AWS deployed HipChat Server instances
• This error will affect all users on the Server that aren't logged in. Some users may not be affected right away

Workaround

Using the developer tools in Chrome, you will see http-bind errors in the Network tab when loading up the app. Notice in the Headers tab that these show a 'Request URL' that is different from 'Request Headers' origin and referer.

For a workaround, log into the HipChat Server command line and run the following command:

sudo dont-blame-hipchat -c "sed -i 's/incorrect-address/correct-hipchat-server-fqdn/g' /etc/hosts"

Make sure to replace incorrect-address and correct-hipchat-server-fqdn with the appropriate values. You may need to do this on /hipchat/config/site.json as well:

sudo dont-blame-hipchat -c "sed -i 's/incorrect-address/correct-hipchat-server-fqdn/g' /hipchat/config/site.json"

After the change is made, please do a full reboot of the server using sudo reboot.