A user can have many attributes. An attribute can have many values. In order to store a map of user attributes, a diff needs to take place in order to determine what attributes need to be added and what need to be replaced. This diff was resolved by removing any existing attribute matches and replacing them with the new attribute values. Basically, it comes down to a DELETE on matching attribute keys and INSERTs for the updated attribute values. Under normal circumstances, this is the preferred approach as it offers cleaner logic.
Concurrently updating a user's attributes under high load can cause simultaneous transactions to perform two INSERTs of the same attribute in succession, violating the uniqueness constraint on the attribute values - due to the read-committed isolation level. This was the error we were experiencing on extranet. The bamboo-svn user performs highly concurrent authentication operations on Crowd. An authentication operation requires updating attributes on the user (eg. last authenticated stamp, etc). And hence the problem.
In Crowd 1.x, we did not experience this problem because we let Hibernate manage our collections, at the cost of table-hops. In Crowd 2.0 we moved the multi-valued attributes from a 2-table model to a 1-table model. In order to be more "effective" with regards to collection mapping, we have implemented manual collection dirty-checking to move away from the DELETE/INSERT approach to an UPDATE and DELETE/INSERT only when absolutely required. This reduces the likelihood of INSERT collisions under high concurrent load.
20 concurrent threads, each with 200 iterations of authentication of the same user, successfully executed against Crowd hooked up to Postgres 8.3. Prior to this commit, 2 concurrent threads would cause INSERT collisions. See PrincipalAuthenticationLoadTest.
This problem was not identified earlier in load tests because our jmeter load tests did not test the specific case of hammering the exact same user under high load.
Bug
Medium
A user can have many attributes. An attribute can have many values. In order to store a map of user attributes, a diff needs to take place in order to determine what attributes need to be added and what need to be replaced. This diff was resolved by removing any existing attribute matches and replacing them with the new attribute values. Basically, it comes down to a DELETE on matching attribute keys and INSERTs for the updated attribute values. Under normal circumstances, this is the preferred approach as it offers cleaner logic.
Concurrently updating a user's attributes under high load can cause simultaneous transactions to perform two INSERTs of the same attribute in succession, violating the uniqueness constraint on the attribute values - due to the read-committed isolation level. This was the error we were experiencing on extranet. The bamboo-svn user performs highly concurrent authentication operations on Crowd. An authentication operation requires updating attributes on the user (eg. last authenticated stamp, etc). And hence the problem.
In Crowd 1.x, we did not experience this problem because we let Hibernate manage our collections, at the cost of table-hops. In Crowd 2.0 we moved the multi-valued attributes from a 2-table model to a 1-table model. In order to be more "effective" with regards to collection mapping, we have implemented manual collection dirty-checking to move away from the DELETE/INSERT approach to an UPDATE and DELETE/INSERT only when absolutely required. This reduces the likelihood of INSERT collisions under high concurrent load.
20 concurrent threads, each with 200 iterations of authentication of the same user, successfully executed against Crowd hooked up to Postgres 8.3. Prior to this commit, 2 concurrent threads would cause INSERT collisions. See PrincipalAuthenticationLoadTest.
This problem was not identified earlier in load tests because our jmeter load tests did not test the specific case of hammering the exact same user under high load.