Plugin Servlets are not recreated after importing configuraton

When importing config, all Jira components get recreated except servlets.
I have a servlet that holds a reference to Jira component. After config import that JiraComponent is not valid anymore, but my old servlet keeps referencing it. All url calls are still handled by old servlet