Currently, there is no possible way to differentiate the issues in backlog or board of a nextgen project with Kanban methodology. The status is not sufficed and hence a JQL cannot be used to filter out the issues. 

Steps - 

1) Use a next gen project in Jira
2) Enable the function "board" and "backlog"
3) Switch to the backlog per navigation
3) Create two issues
4) Move one issue from the lower backlog part to the upper part "board"

5) You can now see one issue being displayed on the board (navigate to board) and one further issue, which is visible in the "backlog" view.

There is no possible way to apply a JQL that can identify the issues in the backlog or board.