We have several spaces that are visible only to users in specific groups, but they show up in searches by anyone.

I went to our Confluence as an unregistered user and did a search for "a*". That returned a list of virtually every page on the server including title, space name, and a summary of the contents.