Opening the link for "Remove Attachments" in a separate tab will remove the attachment straight away, bypassing the confirmation.

This is a particularly serious problem for Windows users, as clicking the middle mouse button on a link will open it in a new tab (tested with IE7 and Firefox3).