Summary

monit will send false scissortail alerts after upgrade from 1.1x to 2.x

Steps to Reproduce

1. Upgrade HipChat Server from 1.1.8 to 2.2.0
2. Run monit status

Expected Results

All processes should be reported as running

Actual Results

The two errors will be t
/etc/monit.d/scissortail.monitrc:2: Program does not exist: '/hipchat/scissortail/current/wrapper.sh'
/etc/monit.d/scissortail.monitrc:3: Program does not exist: '/hipchat/scissortail/current/wrapper.sh'

The two processes scissortail-rqworker and scissortail-gunicorn will be reported as failed:

Process 'scissortail-rqworker'
  status                            Execution failed
  monitoring status                 Monitored
  data collected                    Thu, 02 Mar 2017 11:15:59

Process 'scissortail-gunicorn'
  status                            Execution failed
  monitoring status                 Monitored
  data collected                    Thu, 02 Mar 2017 11:16:29

Workaround

The root cause being the presence of a stale /etc/monit.d/scissortail.monitrc in the server, the workaround would be:

1. Move the file out of the way:

   sudo dont-blame-hipchat
   mv -v /etc/monit.d/scissortail.monitrc /home/admin/scissortail.monitrc.bak # Pick any other location if needed
   

2. Restart monit:

   service monit restart
   

3. Validate the errors are gone:

   monit status