The mail protocol does not get set against the mail server in memory.
Restarting Bamboo will fix the problem.