Allow issue security based on "Participants" or "Watchers"

I have an issue security scheme for "Reporter and Assignee Only".

Where this breaks down, is when a manager assigns a private issue to someone. That person then updates information, but wants to assign it back to their manager to force their manager to respond to their comment, the person can no longer see the issue.

Also, sometimes the issue is private, but you want to dynamically bring a random person into that issue.

What I really want is, "let anyone who has been assigned the issue at some point be alloweded to see that issue". Which could work off of the "Participants" field that is in the toolkit.

Or, "Let anyone that has been added as a Watcher be able to view this issue"