Remaining estimate can be entered for issue types that have time tracking disabled via field configuration

We are putting several issue types onto a rapidboard and estimate them via story points. However, we also want to be able to track time for particular issue types (e.g. technical tasks). If we enable Time Tracking on the board, we can enter a remaining estimate for stories even though time tracking is disabled for stories in the field configuration.

This leads to inconsistency and confuses our users. Is there any way of showing this field only for the correct issue types?