It is not possible to see a list of all boards connected to a JIRA project. Agile | Manage Boards show all boards in the system.

But it is not a robust way because a board can be connected to several projects through it's filter, but only connected to e.g. one project, or none in this list.

What I miss is a section under a project's Overview or Administration, where it could be nice to see all the boards that are connected to the specific project.