Summary

Only a current team member can remove a member. 

How to reproduce

1. Access "People" >> "Search people and teams".
2. Find a group that you won't be a member of.
3. Even a site-admin won't be able to remove a team member.

Improvement suggested

Site-admin should be able to remove a team member, or at least we should grant permission to a group to edit groups.

More asks from a similar request: 

As stated in the article "Create and manage an Atlassian team" (under "Add or remove members of a team"), all members of your team have the same permissions, so any member of the team can invite new team members or remove other team members.

As a site-admin, it would be great if you could remove team member without having to join that team.