Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-4413

Unique key constraints not working for Embedded Crowd table cwd_membership

      Crowd can generate duplicate rows Crowd for memberships. This causes exceptions when Stash tries to sync with crowd.

      Stash has constraints in cwd_membership.

      org.hibernate.NonUniqueResultException: query did not return a unique result: 2
      	at org.hibernate.internal.AbstractQueryImpl.uniqueElement(AbstractQueryImpl.java:914) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
      	at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:396) ~[hibernate-core-4.2.5.Final.jar:4.2.5.Final]
      	at com.atlassian.crowd.dao.membership.MembershipDAOHibernate.findInternalMembership(MembershipDAOHibernate.java:356) ~[crowd-persistence-hibernate4-2.7.1-rc.jar:na]
      	at com.atlassian.crowd.dao.membership.MembershipDAOHibernate.isUserDirectMember(MembershipDAOHibernate.java:211) ~[crowd-persistence-hibernate4-2.7.1-rc.jar:na]
      	...
      
      

      Workaround

      Remove and reconnect the affected Crowd directory. More details:
      https://confluence.atlassian.com/display/STASHKB/Unable+to+sync+crowd+user+directory+-+query+did+not+return+a+unique+result

      Additional note

      The fix for this issue inserted the uk_mem_dir_parent_child into a table on our database. After upgrading to versions above that fix, you should get the duplicate records removed from the database and all fixed.

      If your database was in bad shape you could still have a few duplicate records that hadn't been deleted during the upgrade task. This will get your synchronisation failing with the following error:

      2016-08-25 18:07:57,939 ERROR [Caesium-1-4] o.h.e.jdbc.spi.SqlExceptionHelper ERROR: duplicate key value violates unique constraint "uk_mem_dir_parent_child"
      2016-08-25 18:07:57,963 ERROR [Caesium-1-4] c.a.c.d.DbCachingDirectoryPoller Error occurred while refreshing the cache for directory [ 2457601 ].
      org.hibernate.exception.ConstraintViolationException: could not execute statement
      
      

      If you're hitting that, please proceed with the following steps:

      1. Create a copy of your External Directory connection (i.e. LDAP, JIRA, etc) in Administration >> User Directories
      2. Delete the old one
      3. The issue won't return as the constraint will never allow Crowd's bug manifest again.

            [BSERV-4413] Unique key constraints not working for Embedded Crowd table cwd_membership

            Thanks Bryan, issue opened

            Idan Bidani added a comment - Thanks Bryan, issue opened

            idan.bidani,

            That error means the unique constaints are working. Put differently, it shows that this issue is fixed. Previously, the duplicate insert would have just worked, and corrupted the data.

            That error should be benign (it shouldn't happen at all, of course, but when it does it should be benign). Are you seeing it often? When does it happen? You might want to open a ticket at support.atlassian.com, where you can share details and logs with us privately, rather than trying to diagnose the problem here.

            Best regards,
            Bryan Turner
            Atlassian Bitbucket

            Bryan Turner (Inactive) added a comment - idan.bidani , That error means the unique constaints are working. Put differently, it shows that this issue is fixed. Previously, the duplicate insert would have just worked, and corrupted the data. That error should be benign (it shouldn't happen at all, of course, but when it does it should be benign). Are you seeing it often? When does it happen? You might want to open a ticket at support.atlassian.com, where you can share details and logs with us privately, rather than trying to diagnose the problem here. Best regards, Bryan Turner Atlassian Bitbucket

            I'm seeing this error on Bitbucket 5.6.2 with Postgres

            Any idea what could cause it to happen if this was fixed?

            Thanks

            < 2018-09-17 10:18:28.039 EDT > ERROR:  duplicate key value violates unique constraint "uk_mem_dir_parent_child"
            < 2018-09-17 10:18:28.039 EDT > DETAIL:  Key (directory_id, lower_child_name, lower_parent_name, membership_type)=(4325377, <<REAL_USERID>>, <<REAL_DL_NAME>>, GROUP_USER) already exists.
            < 2018-09-17 10:18:28.039 EDT > STATEMENT:  insert into cwd_membership (parent_id, child_id, membership_type, group_type, parent_name, lower_parent_name, child_name, lower_child_name, created_date, directory_id, id) values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
            

            Idan Bidani added a comment - I'm seeing this error on Bitbucket 5.6.2 with Postgres Any idea what could cause it to happen if this was fixed? Thanks < 2018-09-17 10:18:28.039 EDT > ERROR: duplicate key value violates unique constraint "uk_mem_dir_parent_child" < 2018-09-17 10:18:28.039 EDT > DETAIL: Key (directory_id, lower_child_name, lower_parent_name, membership_type)=(4325377, <<REAL_USERID>>, <<REAL_DL_NAME>>, GROUP_USER) already exists. < 2018-09-17 10:18:28.039 EDT > STATEMENT: insert into cwd_membership (parent_id, child_id, membership_type, group_type, parent_name, lower_parent_name, child_name, lower_child_name, created_date, directory_id, id) values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)

            Brent P added a comment -

            heshanm - We updated the priority. Note that this issue was already fixed in Stash 3.7.0.

            Brent P added a comment - heshanm - We updated the priority. Note that this issue was already fixed in Stash 3.7.0.

            Atlassian,

            Can you please let me know how you guys prioritize this as a minor bug, since one or more users are unable to access the system due to sync fail.

            Cheers,
            heshan

            Heshan Manamperi added a comment - Atlassian, Can you please let me know how you guys prioritize this as a minor bug, since one or more users are unable to access the system due to sync fail. Cheers, heshan

            As part of the fix a unique key constraint has been added to `cwd_membership`.

            Justin Thomas added a comment - As part of the fix a unique key constraint has been added to `cwd_membership`.

            Thanks Bjarni, that's perfect.

            looking at the database table definitions for this it looks like a similar (but not the same) problem. I've created STASH-5244 to track this issue.

            cheers,
            James

            James Gorman added a comment - Thanks Bjarni, that's perfect. looking at the database table definitions for this it looks like a similar (but not the same) problem. I've created STASH-5244 to track this issue. cheers, James

            Hi James,

            Here is what I confirm:

            • JIRA tables are OK
            • Stash cwd_user contains the duplicates and these users can not log in
            • Stash cwd_membership did not contain any entries for the one of the "bad entries" (like I had the same user twice, one of these users had entries in the membership table but the other one not)
            • I'm running Stash 2.12.1
            • I see the same error in my log as is reported (that is how I found this issue)

            I fixed this myself by simply deleting the "second user" from the cwd_user table in Stash.

            Hope this helps,

            -Bjarni

            Bjarni Thorbjornsson added a comment - Hi James, Here is what I confirm: JIRA tables are OK Stash cwd_user contains the duplicates and these users can not log in Stash cwd_membership did not contain any entries for the one of the "bad entries" (like I had the same user twice, one of these users had entries in the membership table but the other one not) I'm running Stash 2.12.1 I see the same error in my log as is reported (that is how I found this issue) I fixed this myself by simply deleting the "second user" from the cwd_user table in Stash. Hope this helps, -Bjarni

            Hi Bjarni,

            could you please confirm that what you are seeing is:

            • JIRA tables are fine (no duplicates in cwd_user or cwd_membership)
            • Stash cwd_user contains duplicates (as above)
              • These users cannot log in
            • Stash cwd_membership does not contain duplicates

            If this is the case it looks like a different problem and if you haven't fixed it yourself we should open a support ticket for you. If you have fixed it I'll create a new bug report to get this fixed.

            Some related questions:

            • what version of Stash are you using?
            • are you seeing error logging related to the failed login?

            We will hopefully have a fix to the problem originally on this ticket soon.

            cheers,
            James

            James Gorman added a comment - Hi Bjarni, could you please confirm that what you are seeing is: JIRA tables are fine (no duplicates in cwd_user or cwd_membership ) Stash cwd_user contains duplicates (as above) These users cannot log in Stash cwd_membership does not contain duplicates If this is the case it looks like a different problem and if you haven't fixed it yourself we should open a support ticket for you. If you have fixed it I'll create a new bug report to get this fixed. Some related questions: what version of Stash are you using? are you seeing error logging related to the failed login? We will hopefully have a fix to the problem originally on this ticket soon. cheers, James

            One more thing,

            When I execute the query above on the Stash database, I do get several duplicates but when I execute the same query on the JIRA database nothing shows.

            JIRA is synced with AD and Stash is synced with JIRA.

            Thanks again,

            -Bjarni

            Bjarni Thorbjornsson added a comment - One more thing, When I execute the query above on the Stash database, I do get several duplicates but when I execute the same query on the JIRA database nothing shows. JIRA is synced with AD and Stash is synced with JIRA. Thanks again, -Bjarni

              jthomas@atlassian.com Justin Thomas
              jgorman James Gorman
              Affected customers:
              7 This affects my team
              Watchers:
              15 Start watching this issue

                Created:
                Updated:
                Resolved: