Details
-
Bug
-
Resolution: Fixed
-
Low
-
None
-
None
-
Severity 3 - Minor
Description
Problem
Upgrading HipChat Server from 1.4.2 to 1.4.3 fails during the Chef run, complaining that it cannot install the lvm2 package.
Trying to force an upgrade after the failure may throw the following errors:
Calculated sha512sum: b55f7c1b139ac6cba89eef57164d76ed60d70dcc4ecab70fbfe4fe7c5364ca51c6b64e9e9d423ecdcb5231d658847372470d90e585c678bf0e032e924f0cf395 hcs-1.4.3-2016.07.11.082020.tar.gz coercing to Unicode: need string or buffer, NoneType found Updating system configuration, this may take a few minutes... There was an unexpected error: Command '['sudo', '/opt/atlassian/hipchat/sbin/cs']' returned non-zero exit status 1
The following is seen in /var/log/chef.log which indicates that the upgrade did not complete successfully due to the failure to fetch the lvm2 package from the on-disk Ubuntu archive. Note the FILE NOT FOUND error below for lvm2_2.02.98-6ubuntu2_amd64.deb
[2016-07-22T05:47:22+00:00] ERROR: bash[apt_dist_upgrade_non_interactive] (hipchat_server_cleanup::apt_dist_upgrade_non_interactive line 2) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '100' ---- Begin output of "bash" "/tmp/chef-script20160722-2657-1pi26j0" ---- STDOUT: Reading package lists... Building dependency tree... Reading state information... The following packages were automatically installed and are no longer required: consolekit guile-1.8-libs libavcodec53 libavformat53 libavutil51 libboost-date-time1.46.1 libboost-iostreams1.46.1 libboost-program-options1.46.1 libboost-serialization1.46.1 libboost-thread1.46.1 libclass-isa-perl libcloog-ppl0 libdconf0 libgdu0 libgearman6 libgnome-keyring-common libgnome-keyring0 liblcms1 libllvm3.0 liblua5.1-0 liblvm2app2.2 libmagickcore4 libmagickwand4 libmpc2 libodbc1 libpam-ck-connector libpango1.0-0 libpangoxft-1.0-0 libpostproc52 libppl9 libpwl5 libsgutils2-2 libswitch-perl libtcl8.5 libtiff4 libtokyocabinet8 python-boto python-crypto python-m2crypto python-paramiko python-pycurl python-software-properties python-support tcl8.5 tcl8.5-dev udisks Use 'apt-get autoremove' to remove them. The following packages will be upgraded: lvm2 1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Need to get 0 B/470 kB of archives. After this operation, 290 kB of additional disk space will be used. Err file:/file_store/archive/ bintray-trusty-btf/main lvm2 amd64 2.02.98-6ubuntu2 File not found STDERR: E: Failed to fetch file:/file_store/archive/pool/main/l/lvm2/lvm2_2.02.98-6ubuntu2_amd64.deb File not found E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ---- End output of "bash" "/tmp/chef-script20160722-2657-1pi26j0" ---- Ran "bash" "/tmp/chef-script20160722-2657-1pi26j0" returned 100 [2016-07-22T05:47:23+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1) allow custom /etc files
This puts the server in an unknown state and renders HipChat unusable.
Workaround
For users affected by this problem, please follow the steps below.
Log in to the HipChat Server's command line and execute the following commands:
sudo dont-blame-hipchat
cd /file_store/archive/pool/main/l/lvm2/
wget https://s3.amazonaws.com/hipchat-server-stable/utils/lvm2_2.02.98-6ubuntu2_amd64.deb
cs
The last command, cs, will kick off a Chef Solo run and correctly install the previously missing lvm2 package. Once the command completes, please log in to the HipChat clients and confirm things are working again.
