When saving the Announcement Banner configuration with the Status set to "Off" will correctly save the configuration, however when visiting the page again it will not show the saved message and audience.
1. Go to the rate limiting admin page
2. Set the status to "Off"
3. Add some text in the banner message field
4. Save the banner.
You should now see a notification that states the banner has been saved.
5. Refresh the page
The saved banner settings are shown
The banner message is empty (and if changed from the default, the banner audience is reverted to the default value).
No workaround is available. Should one become available this issue will be updated.