-
Suggestion
-
Resolution: Unresolved
-
None
-
1
-
21
-
Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.
NOTE: This suggestion is for Confluence Cloud. Using Confluence Server? See the corresponding suggestion.
Whatever the administrator enters in as the "Group name attribute" must be unique for all groups in the LDAP server. Typically this is the "CN" attribute.
In the event that two different entities in LDAP are found with the same value for that attribute (CN), you receive the following stack trace.
2011-09-01 00:26:43,979 ERROR [QuartzScheduler_Worker-2] [atlassian.crowd.directory.DbCachingDirectoryPoller] pollChanges Error occurred while refreshing the cache for directory [ 142802946 ]. java.lang.IllegalArgumentException: duplicate key: duplicatedGroupName at com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:62) at com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:210) at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:196) at com.google.common.collect.Maps.uniqueIndex(Maps.java:456) at com.atlassian.crowd.directory.ldap.cache.AbstractCacheRefresher.synchroniseMemberships(AbstractCacheRefresher.java:126)
Excluding these duplicates from the sync is not possible due to the fact that we can't control the order in which groups are returned from the LDAP server. If the ordering were to change between syncs, the memberships could keep changing on the Confluence side.
The solution to this problem is to amalgamate memberships for groups which share the same name on the remote server. We should make this an option in the LDAP configuration.
- is blocked by
-
CWD-3227 Merge memberships for groups with duplicate names during LDAP directory sync
- Closed
- is related to
-
CWD-2681 Crowd 2.3.3 does not sync user memberships when multiple LDAP groups have the same name
-
- Closed
-
-
CWD-2504 Synchronization with JIRA/Confluence fail with duplicate entry
-
- Closed
-
-
CWD-2441 Use sAMAccountName attribute for group name by default when using Active Directory
- Closed
-
CONFSERVER-23213 Merge memberships for groups with duplicate names during LDAP directory sync
- Gathering Interest
[CONFCLOUD-23213] Merge memberships for groups with duplicate names during LDAP directory sync
As of Confluence 4.3.1, we will ignore the duplicate group and its members as per CWD-2504, rather than aborting the synchronisation. Sadly, the improvement requested by this issue is still not yet implemented, due to the extra complexities of resolving which group to modify for writable directories, or directories with local groups.
Mark - yes I believe that issue sums it up. I figured I'd ask the question, and to Matt's point, we can certainly change the identifier which I have begun looking into and it might work.
And to be honest, its a small fraction of the groups within our LDAP that have duplicate names. And furthermore, they're usually old legacy groups that we don't even care about unrelated to Confluence. But it just sucks each one kills the sync, and then you have to fix it, sync again, fix the next one, etc... it's not easy to query LDAP for all the dups. I'd rather it find a dup, ignore it and log it, and continue. This would be sufficient in most cases since its not a big deal, and in those where a customer actually cares about the group, they'll be more likely to fix it.
Thanks guys, good discussion.
I'm not sure that's what David was alluding to (usage of a unique identifier in addition to a name).
He mentions using sAMAccountName as key, then suggests a separate Display name for groups:
"And when users search for groups in Confluence, they can obviously search on the more friendly Group Name ..."
When I have discussed using sAMAccountName for group name to a customer previously, they didn't think it was appropriate because it was not the name that the users "were used to seeing".
See JRA-26164
@Matt:
I'm not sure that's what David was alluding to (usage of a unique identifier in addition to a name).
For usernames, we use (by default) sAMAccountName or UID in most places: these are generally constrained to be unique from the LDAP Side.
For groups, we use CN, which is not constrained; though in many of these systems, groups also have UID's and sAMAccountName as fields.
I think David is referring to this inconsistency. Surely there are systems that do not have a constrained unique identifier for group entities, but we could just fall back to an unconstrained identifier for that attribute.
… why is Confluence (or more accurately Crowd) basing its unique identifier for groups off of something in another system that is not guaranteed to be unique?
For better or worse, this is how Confluence, JIRA and Crowd work at the moment. They only support a name for a group, not a separate unique identifier. The group name has to be unique.
Unfortunately, this is unlikely to change in the future, at least on the Confluence side. We have a limited number of people to work on the product, and we aim to keep the products simple as we enhance it. Adding a group unique identifier to the UI in all the places where they appear would take a lot of work and complicate the UI while only helping a very small number of our customers. A similar situation applies to supporting multiple users with different names (the so-called "domains" in Microsoft software).
We offer the ability to configure the group name attribute in our LDAP configuration for this reason. You need to pick an attribute which has a unique value for each group (or filter out the duplicate groups). Until we can resolve this issue, that is the best workaround for this problem.
These are all great points, and I would also be fine with merging the groups as well (although giving us the option to either merge or exclude duplicate groups would be ideal).
If I may ask a rudimentary question; why is Confluence (or more accurately Crowd) basing its unique identifier for groups off of something in another system that is not guaranteed to be unique? That'd be like making your Confluence login ID your first name in LDAP; a bad idea for obvious reasons. So couldn't Crowd use something like sAMAccountName (I know that's AD specific, but it could be something that is known to be unique in whatever type of LDAP Directory you choose) or maybe it could use the DistinguishedName (DN) of the group, which is guaranteed to be unique? And when users search for groups in Confluence, they can obviously search on the more friendly Group Name (which would return multiple results) but they are also shown the DN so they know which one they are actually choosing.
@Matt
I can't see how it would be technically possible to merge groups but not technically possible to make the group empty.
There are customers above on this ticket that have requested the ability to merge the memberships, because they believe this would be the most accurate representation of their LDAP configuration in our applications.
I have to concede that this would really be giving them what they asked for, I guess I was trying to "protect them from themselves".
If we do implement it, I believe it should be an option in the directory configuration, so we're by no means forcing every customer into using this behaviour.
This would make me feel a lot more comfortable if merge is off by default.
I don't believe this is a big security concern. LDAP is a trusted source of membership information, and is generally difficult to change in our customers' organisations.
I am not so much worried about deliberate attack as an accidental escalation of privileges.
eg imagine a customer and under some subtree lived a group with CN=admins that contained all the system administrators for their intranet Confluence server and another subtree lived a group with CN=admins that contains the 2000 shop administrators for their various retail outlets.
The real problem is that the sync dies at this point and doesn't provide meaningful log messages.
It does provide a meaningful log message in recent versions of Crowd. That has already been fixed.
Letting the rest of the synch complete - only fail on the invalid group names
If it's possible to do this easily with the current sync code, I wouldn't disagree. However, the Crowd guys didn't seem to believe it was possible when we discussed all these possibilities with them.
By contrast, the merging of memberships didn't seem too complicated - it just requires a bit of an understanding of Crowd's internals. It also mimics what we do with group membership across multiple directories in the application.
We actually explored this possibility (merging) some 6 months or more ago and it was rejected by JIRA and Crowd because of security concerns. ... Then later someone adds a different group to LDAP that happens to get mapped to the "blah" groupname in an Atlassian app, and suddenly there is a bunch of people with elevated privileges.
I don't believe this is a big security concern. LDAP is a trusted source of membership information, and is generally difficult to change in our customers' organisations.
There are customers above on this ticket that have requested the ability to merge the memberships, because they believe this would be the most accurate representation of their LDAP configuration in our applications.
If we do implement it, I believe it should be an option in the directory configuration, so we're by no means forcing every customer into using this behaviour.
It is also important to note Tim's comment from Support:
The original report asked for a graceful handling (not necessarily merging)
The real problem is that the sync dies at this point and doesn't provide meaningful log messages.
Support originally raised this issue as "Gracefully handle duplicate key for group name during directory sync".
That means:
- Letting the rest of the synch complete - only fail on the invalid group names
- Putting meaningful error messages in the logs with full information on which groups have failed to synchronise.
I have to say that I am concerned with the security implications of merging group memberships.
We actually explored this possibility (merging) some 6 months or more ago and it was rejected by JIRA and Crowd because of security concerns.
eg you have an LDAP group called "blah" with certain members and you add, say, admin rights to this group.
Then later someone adds a different group to LDAP that happens to get mapped to the "blah" groupname in an Atlassian app, and suddenly there is a bunch of people with elevated privileges.
Given that we can't implement "first in wins" without remembering the full DN, I think a safer approach is to call this group invalid and delete it from the synch (ie no-one wins).
IF the customer really wants this particular group (which group did they want anyway?) - they can setup "LDAP with local groups" and manually maintain the group up until sanity is restored (or "split" their LDAP Directory into two separate User Directories for different branches of the LDAP tree)
dhergert - yes, it is the same issue. We've improved the error message with our previous work on this issue.
Are none of the above workarounds applicable in your situation? You currently need to have unique group names to synchronise LDAP with Confluence, so you can choose another attribute which is unique or limit the LDAP search to include just one of the affected groups as described above.
The remaining work includes better testing and figuring out how to write membership changes back to the LDAP server.
edalgliesh, dealing with updates to multiple LDAP groups that Confluence (Crowd) is amalgamating should be out of scope for this fix. I think we could add some switching logic so this option is only available in read-only directories. In Crowd terms, that means directories where the group modification permission is not enabled.
Whoever continues work on this should confirm this approach with the Crowd devs.
We are also experiencing this in Confluence, and its preventing the successful sync with the User Directory. It would be nice if it didn't bomb complete and instead logged a warning, but then continued processing. That way at least the rest of the groups get added.
I am seeing a slightly different error, albeit still related to Crowd, that seems to be documented in CWD-2796. Is it the same issue?
2012-05-14 10:22:49,041 ERROR [scheduler_Worker-4] [atlassian.crowd.directory.DbCachingDirectoryPoller] pollChanges Error occurred while refreshing the cache for directory [ 360450 ]. com.atlassian.crowd.exception.OperationFailedException: Unable to synchronise directory: duplicate groups with name 'Communications' at com.atlassian.crowd.directory.ldap.cache.AbstractCacheRefresher.synchroniseMemberships(AbstractCacheRefresher.java:133) at com.atlassian.crowd.directory.ldap.cache.AbstractCacheRefresher.synchroniseAll(AbstractCacheRefresher.java:44) at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAll(UsnChangedCacheRefresher.java:223) at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:621) at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:63) at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:50) ...
I have to table this issue temporarily, hopefully only until next iteration when we should have more meat-flavoured code monkeys to work on it.
I've committed some changes on the CONF-23213 branch of Crowd, which merges groups together and some corresponding changes on the 1.6-duplicate-group branch of embedded crowd to allow configuration. I've get to write any integration tests for this, and the work should be considered nowhere near complete. These branches as they currently stand should not be considered fit for use. The remaining work includes better testing and figuring out how to write membership changes back to the LDAP server. The case where nested groups are used is particularly gnarly.
Thanks for all your feedback on this issue so far. The error message has been updated in Confluence 4.2 to make it clear that duplicate groups are the problem. However, we don't believe this completely solves the problem. We've done some investigation of this issue and discussed it several times with the development teams at Atlassian.
We currently have four possible approaches for customers who see this issue when connecting Confluence with their LDAP server:
- Restrict the LDAP tree which is searched by Confluence. You can use a more specific "base DN" in your LDAP directory configuration to exclude parts of the tree that contain duplicate names.
- Filter out the affected groups. You can specify a "group filter" in your LDAP directory configuration, such as those described in How to write LDAP search filters, to exclude the groups which have duplicate names.
- Disable referrals if the affected groups are across multiple servers. Often in an Active Directory forest, duplicate group names will appear across multiple servers. Disabling the "follow referrals" setting in your directory configuration will prevent those groups from other servers clashing with those in the main directory.
- Use an attribute which is unique as the group name. Most people on this thread are using the 'cn' attribute as the group name, which happens to not be unique for the given LDAP server. On some LDAP servers, there may be another attribute which can be used as the unique group name in Confluence.
Unfortunately, the simple patch proposed above by Mario Wörndl, and attached by Adam, has serious problems which mean we cannot ship this as a solution to the problem. If two groups exist with the same name in LDAP, it is not certain which group memberships will be retrieved when synchronising with LDAP. In the worst case, the list of memberships for a duplicated group could flip-flop between the two groups depending on the behaviour of the sync each time. We'll be removing the patch and sample code above for this reason.
The only proper fix we have identified for this problem is to "amalgamate" or "combine" memberships for all groups which share the same name in LDAP. Steve Pribyl suggested this above, and we believe that this is the only viable option. Versions of Confluence prior to 3.5 may have done this inadvertently, but it was never something we intended to support. We will update this issue to reflect that this is our proposed solution to this problem, and that we would like to implement this in the future. We're sorry to say that we don't have time to implement this at the moment. Please vote for the issue if you haven't already done so.
Some people above requested support for duplicate group names inside Confluence. We don't currently have any plans to implement this. Our user management system is already very complicated – it is one of the most common reasons Confluence customers contact support – and we believe that supporting groups with the same name would complicate things much further.
In summary, if your LDAP server contains multiple groups with the same name, you will need to take one of the approaches above to select which group you wish to include in Confluence, or vote for this issue to support amalgamation for multiple LDAP groups with the same name.
Edith Tom & Matt Ryall
Confluence Development
I have worked around the problem by not include the whole AD tree by using the group . This means that not all of our groups are being imported. This of course is impacting the usability and security of confluence. While this is functional it is ultimately not acceptable. Add to this the other issue with editing we are looking at alternative solutions.
Please fix the code to allow duplicate group names. Duplicate group names are possible in pretty much any ldap/AD implementation.
Most linux implementations "add" the duplicate groups together.
We worked around this problem as follows:
Code removed by Edith Tom. Please see [my comment below
Actually it would be better to solve the problem in this.groupListFuture.get() / this.roleListFuture.get()
This solution only removes groups with duplicate name but does not allow them. An warning points to the removed group.
Hi
This fix in important for us, because our Ad forest does indeed hold duplicated group names (fx: 'Users' is a common group name). The workaround suggested to swallow and log the error is not enough for us, unless we reorganize our whole Ad structure.
The fix in Crowd does not resolve this issue, instead of merging, duplicate groups are dropped and logged, to allow synchronisation to continue without failing. Reopening this issue to reflect that.