In order to allow respective managers to manage their own projects, without having full administrator rights, one can set the Permission Scheme of projects to give rights to Administer Project to the Project Role (Administrators).

While this enables them to press the 'Administration' button, they are still unable to View and Edit the project roles themselves. I.e. they are unable to add people to work on their project.