A site open to anonymous users allows former users from the site to authenticate and make changes to third-party application macro settings

Issue Summary

Users removed from a site that is open to anonymous users are allowed to authenticate to the site.
They get an error they don't have access to the site at first, but if they try again, they are allowed to login into the site and keep authenticated there.

While they are logged in, other spaces not open to anonymous users are not visible, however, some other third-party macros under the menu APPS in the navigation top bar, display more options to them and allow these users not part of the site to access the third-party app configurations.

One of the examples reported is the Handy macro (third party), which allows a former user to access the site (open to anonymous) to delete a configuration for the app.

Steps to Reproduce

1. Create a test site
2. Install a third-party app for example Handy Macros app (part of the testing)
3. Invite a test account as a licensed user
4. Log in with this account and access some content (open only to the licensed user)
5. As the test user, check the navigation bar > Apps > Handy macro
6. As the admin: Remove the account from the site 
7. As an admin: Open the site to anonymous (Site and space)
8. Make sure to have some spaces not open to anonymous users to validate testing
9. In a new incognito window, access the test site as an anonymous user
10. Try to access any content not open to anonymous
11. Go to the navigation bar > Apps (Try to find the handy macro)
12. Still, on incognito, try to log in with the test user just removed from the site
13. When hitting the error you don't have access, try again
14. After logging in and seeing the content open only to anonymous, click on the navigation bar > Apps and try to access the Handy macro options

Expected Results

The former user would only have access to the anonymous access content in Confluence and nothing else or not be able to login into the site as they no longer have a license/get an error message.

Actual Results

Former users logged in can only see content created open to anonymous users as expected, but some (not all apps) third-party apps options are available for changes when those former users are logged in. (Anonymous users don't have this option, only licensed users)

Workaround

If users only revoked site access/removed licenses, they are blocked from accessing the site when they try to log in, even if the site is open to anonymous, blocking them from making changes to any options for third-party macros.

Suspended/removed license users still can access content for sites open to anonymous users through an incognito window, while unauthenticated.