Certain teams use certain spaces. more as a daily logging thing, which pops up on search due to high content.
Does not make that sense to call it unsuited either looking at content.
Irrespective of our usage. this certainly is a must have feature for Confluence in my opinion.

+1 looking for this feature, but it appears there are paid add-ons someone mentioned above?  

This would be usable eg. in case of personal spaces.
Personal spaces contain often pre-plans or ideas which are not ready to be found as part of important things searching.

I think Hans wanted to mention Confluence Premium plan(online based).

Hans - Can you share the add-ons you mentioned?

How difficult would it be, seriously, to implement a robots.txt type file? The point is that this is trivial functionality, and users should not be expected to buy yet another random nickel-and-dime add-on to get it.  Fancy imports and exports, sure. Page theming, yeah, OK.  Workflows, OK. But the Confluence index update equivalent of robots.txt?  Oh, please, it's basic functionality.

Looking further ....... this is never going to happen because there are paid add-ons to achieve this  . It's al about politics ......

+1

I'm very surprised that this feature doesn't already exist. 

It would be extremely useful to exclude specific topics from the internal search feature. 

The lack of an "unlisted" property on pages or even whole spaces — which excludes them Search results — has been a major pain point for me at three different major startups over the course of five years. It continues to be for us right now. 

Stakeholders become immediately frustrated with navigating page trees in Confluence, and shut down when the Search results are indecipherable (because years of content is unintelligibly piled onto the results page). This has led to Confluence being infamously impossible to navigate across all teams at every company I've touched — and to be frank, it makes Confluence near-useless to developers, who are touchy about process and don't tend to dig for anything.

You must, must update the community with confirmation that you're hearing us on this.  Dropping this comment on CONF-33161 as well since either feature would be so deeply impactful.