At the moment, all site related audit log entries do not explicitly denote which site was affected by that action.

Example:

• Remove someone from your site and check your audit logs, the message displayed will be "Removed someone@somedomain.com from site".

The suggestion is to include the site name as reference to that text: "Removed someone@somedomain.com from the site mysite.atlassian.net" and allow filtering on this new parameter.