Upgrade from 5.1.x to 5.3 fails on SQL Server on alter table FOLLOW_CONNECTIONS add constraint FK_FOLLOW_CONNECTIONS_FOLLOWER

This seems similiar to https://confluence.atlassian.com/pages/viewpage.action?pageId=393904718 but i checked the entry in our cfg.xml and it matches the value mentioned

2013-10-19 08:30:51,877 INFO [main] [confluence.upgrade.upgradetask.UserKeyReferenceUpgradeTask] doUpgrade Completed update of username references.
2013-10-19 08:30:52,299 ERROR [main] [hibernate.tool.hbm2ddl.SchemaUpdate] execute Unsuccessful: alter table FOLLOW_CONNECTIONS add constraint FK_FOLLOW_CONNECTIONS_FOLLOWER foreign key (FOLLOWER) references user_mapping
2013-10-19 08:30:52,314 ERROR [main] [hibernate.tool.hbm2ddl.SchemaUpdate] execute Column 'user_mapping.user_key' is not the same data type as referencing column 'FOLLOW_CONNECTIONS.FOLLOWER' in foreign key 'FK_FOLLOW_CONNECTIONS_FOLLOWER'.
2013-10-19 08:30:52,314 ERROR [main] [hibernate.tool.hbm2ddl.SchemaUpdate] execute could not complete schema update
java.sql.SQLException: Column 'user_mapping.user_key' is not the same data type as referencing column 'FOLLOW_CONNECTIONS.FOLLOWER' in foreign key 'FK_FOLLOW_CONNECTIONS_FOLLOWER'.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368)