Bamboo 3.3 does not respect @RequiresRestart

I tried bumping UPM to support Bamboo 3.3 from 3.1, and we got some failing tests with regards to the tests of the @RequiresRestart. Everything worked in Bamboo 3.1, but there's something wrong in Bamboo 3.3.

Note that this only happens when I'm installing a new plugin with a module that requires restart. But it works as expected when "upgrading" a plugin (i.e., @RequiresRestart is honored)