Issue Time Spent shows time that should not be added, because some worklogs are not Viewable By given user

When I add a worklog to an issue, I select it to be Visible By Developers or Administrators, which does not show that worklog to a user with no Developer permissions. However Time Spent for an issue does show all the time included. It seems like a bug, but may be intentional depending on the criticality of this. For our case, it is critical not to show illegitimate user how many hours were spend and how on a given issue.