Administration Search does not work for freshly installed plugin - JIRA restart required

Only after JIRA restart, I see my entries in Administration Search (G, G).
Refresh, logout/login does not help.

Lack of plugin reloadability strikes back?
Marking it as major because such bug is for me a PR fail (we claim JIRA now is fully reloadable wrt plugins, right?)