i have a different scenario, where this feature completely messed up the board, because it actually shows "done and updated in last x weeks", let me explain:
We had a thousand of closed tickets, some of them were "Done" others "Cancelled", etc.
Yesterday i had to bulk change more than 200 of these tickets, changing "Assignee" and/or "Reporter", i didn't changed any related to its status.
Now i have more than 200 (and the ones that already were) in the Done Column.
And the worst, i can't remove them from there, because i can't change the Query for Done Column.
So, my suggestion is to change the definition of "Done" query to search for trully DONE tickets in the last x weeks (resolution date), not updated tickets AND done.
Our use case: Tempo Timesheets regularly deletes worklogs as part of a data retention policy we have set. This adds issues to the board as these issues are "updated" according to Jira.