Uploaded image for project: 'Crowd Data Center'
  1. Crowd Data Center
  2. CWD-4714

Enable Incremental Sync on All Cases, For Application to Crowd Syncs

XMLWordPrintable

    • Icon: Suggestion Suggestion
    • Resolution: Duplicate
    • None
    • None
    • None

    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Incremental sync for an application (client application such as Confluence synchronising from Crowd), is disabled if the following condition is met:

        private void assertIncrementalSynchronisationIsAvailable(List<Directory> activeDirectories) throws IncrementalSynchronisationNotAvailableException
    {
        for (Directory directory : activeDirectories)
        {
            // No events are generated for cacheable directories that are not cached
            if (isFalse(toBooleanObject(directory.getValue(DirectoryProperties.CACHE_ENABLED))))
            {
                throw new IncrementalSynchronisationNotAvailableException("Directory '" + directory.getName() + "' is not cached and so cannot be incrementally synchronised");
            }
        }
    }

      Where com.atlassian.crowd.directory.sync.cache.enabled is set to false, for the directory's attribute in cwd_directory_attribute. Considering that users and groups for an uncached directory is still cached in the DB anyway, it seems that this check does nothing but to create sync performance issues for customers with delegated directories (it will always fall back to full sync). We should remove this check altogether, or at least, set cache.enabled to always true, for delegated directories

            [CWD-4714] Enable Incremental Sync on All Cases, For Application to Crowd Syncs

            Katherine Yabut made changes -
            Workflow Original: JAC Suggestion Workflow [ 3388973 ] New: JAC Suggestion Workflow 3 [ 3630352 ]
            Status Original: RESOLVED [ 5 ] New: Closed [ 6 ]
            Monique Khairuliana (Inactive) made changes -
            Workflow Original: Simplified Crowd Development Workflow v2 [ 1397897 ] New: JAC Suggestion Workflow [ 3388973 ]
            Issue Type Original: Improvement [ 4 ] New: Suggestion [ 10000 ]
            Status Original: Closed [ 6 ] New: Resolved [ 5 ]

            Lukasz Pater added a comment -

            To clarify: this is not about delegated or connector directories directly (delegated directories don't synchronise as they don't have a cache, connector directories with caches enabled do incremental synchronisation for ActiveDirectory if enabled, full for others).

            This is about the case of a Crowd directory that synchronising with a Crowd instance, that's using either delegated/connector directories. Currently only Crowd directories connected as applications that use cached directories can do incremental synchronisation, other cases always do full synchronisation.

            Lukasz Pater added a comment - To clarify: this is not about delegated or connector directories directly (delegated directories don't synchronise as they don't have a cache, connector directories with caches enabled do incremental synchronisation for ActiveDirectory if enabled, full for others). This is about the case of a Crowd directory that synchronising with a Crowd instance, that's using either delegated/connector directories. Currently only Crowd directories connected as applications that use cached directories can do incremental synchronisation, other cases always do full synchronisation.

            Andriy Yakovlev [Atlassian] added a comment -

            So quick summary:

            • delegated directories - is not cached and doesn't support INCREMENTAL synchronisation, so FULL sync will be used always (see CWD-3717).
            • connector directories that are cached - support INCREMENTAL synchronisation
            • connector directories that are not cached (see Configuring Caching for an LDAP Directory) - doesn't support INCREMENTAL synchronisation, so FULL sync will be used always

            Andriy Yakovlev [Atlassian] added a comment - So quick summary: delegated directories - is not cached and doesn't support INCREMENTAL synchronisation, so FULL sync will be used always (see CWD-3717 ). connector directories that are cached - support INCREMENTAL synchronisation connector directories that are not cached (see Configuring Caching for an LDAP Directory ) - doesn't support INCREMENTAL synchronisation, so FULL sync will be used always

            Foo Sim (Inactive) added a comment -

            Makes sense, thanks for the detailed explanation lpater .

            Foo Sim (Inactive) added a comment - Makes sense, thanks for the detailed explanation lpater .
            Lukasz Pater made changes -
            Resolution New: Duplicate [ 3 ]
            Status Original: Open [ 1 ] New: Closed [ 6 ]
            Lukasz Pater made changes -
            Link New: This issue duplicates CWD-3717 [ CWD-3717 ]
            Lukasz Pater made changes -
            Description Original: Incremental sync for an application (client application such as Confluence synchronising from Crowd), is disabled if the following condition is met:
            {code}
              private void assertIncrementalSynchronisationIsAvailable(List<Directory> activeDirectories) throws IncrementalSynchronisationNotAvailableException
                {
                    for (Directory directory : activeDirectories)
                    {
                        // No events are generated for cacheable directories that are not cached
                        if (isFalse(toBooleanObject(directory.getValue(DirectoryProperties.CACHE_ENABLED))))
                        {
                            throw new IncrementalSynchronisationNotAvailableException("Directory '" + directory.getName() + "' is not cached and so cannot be incrementally synchronised");
                        }
                    }
                }
            {code}
            Where {{com.atlassian.crowd.directory.sync.cache.enabled}} is set to *false*, for the directory's attribute in cwd_directory_attribute. Considering that users and groups for an uncached directory is still cached in the DB anyway, it seems that this check does nothing but to create sync performance issues for customers with delegated directories (it will always fall back to full sync). We should remove this check altogether, or at least, set cache.enabled to *always true*, for *delegated directories*

            h3.Workaround
            Run this query in the Crowd DB:
            {code}
            UPDATE cwd_directory_attribute SET attribute_value = 'true' WHERE attribute_name = 'com.atlassian.crowd.directory.sync.cache.enabled' AND directory_id = <directoryid>;
            {code}
            Where <directoryID> is the directory ID of the delegated directory.

            This will require a Crowd restart             		New: Incremental sync for an application (client application such as Confluence synchronising from Crowd), is disabled if the following condition is met:
            {code}
              private void assertIncrementalSynchronisationIsAvailable(List<Directory> activeDirectories) throws IncrementalSynchronisationNotAvailableException
                {
                    for (Directory directory : activeDirectories)
                    {
                        // No events are generated for cacheable directories that are not cached
                        if (isFalse(toBooleanObject(directory.getValue(DirectoryProperties.CACHE_ENABLED))))
                        {
                            throw new IncrementalSynchronisationNotAvailableException("Directory '" + directory.getName() + "' is not cached and so cannot be incrementally synchronised");
                        }
                    }
                }
            {code}
            Where {{com.atlassian.crowd.directory.sync.cache.enabled}} is set to *false*, for the directory's attribute in cwd_directory_attribute. Considering that users and groups for an uncached directory is still cached in the DB anyway, it seems that this check does nothing but to create sync performance issues for customers with delegated directories (it will always fall back to full sync). We should remove this check altogether, or at least, set cache.enabled to *always true*, for *delegated directories*

            Lukasz Pater added a comment -

            fsim I believe there are two cases you need to consider:

            • delegated directories - in that case users that have logged in (and their groups) are in fact cached in the directory. One could imagine enabling incremental sync for such dirs, but that needs some testing before it can be recommended to customers/enabled globally - this was previously reported and is tracked as CWD-3717
            • connector directories that are not cached - for these there is no data cached in the local db - all requests are handled directly by the remote server. For these we don't track the information about the changes in the directory, which are later used for incremental synchronisation, so if a Crowd server has one of those we can only do a full sync.

            I'll close this as duplicate, and remove the suggested workaround, as it might have other side effects for customers.

            Lukasz Pater added a comment - fsim I believe there are two cases you need to consider: delegated directories - in that case users that have logged in (and their groups) are in fact cached in the directory. One could imagine enabling incremental sync for such dirs, but that needs some testing before it can be recommended to customers/enabled globally - this was previously reported and is tracked as CWD-3717 connector directories that are not cached - for these there is no data cached in the local db - all requests are handled directly by the remote server. For these we don't track the information about the changes in the directory, which are later used for incremental synchronisation, so if a Crowd server has one of those we can only do a full sync. I'll close this as duplicate, and remove the suggested workaround, as it might have other side effects for customers.
            Foo Sim (Inactive) made changes -
            Description Original: Incremental sync for an application (client application such as Confluence synchronising from Crowd), is disabled if the following condition is met:
            {code}
              private void assertIncrementalSynchronisationIsAvailable(List<Directory> activeDirectories) throws IncrementalSynchronisationNotAvailableException
                {
                    for (Directory directory : activeDirectories)
                    {
                        // No events are generated for cacheable directories that are not cached
                        if (isFalse(toBooleanObject(directory.getValue(DirectoryProperties.CACHE_ENABLED))))
                        {
                            throw new IncrementalSynchronisationNotAvailableException("Directory '" + directory.getName() + "' is not cached and so cannot be incrementally synchronised");
                        }
                    }
                }
            {code}
            Where {{com.atlassian.crowd.directory.sync.cache.enabled}} is set to *false*, for the directory's attribute in cwd_directory_attribute. Considering that users and groups for an uncached directory is still cached in the DB anyway, it seems that this check does nothing but to create sync performance issues for customers with delegated directories (it will always fall back to full sync). We should remove this check altogether, or at least, set cache.enabled to *always true*, for *delegated directories*

            		New: Incremental sync for an application (client application such as Confluence synchronising from Crowd), is disabled if the following condition is met:
            {code}
              private void assertIncrementalSynchronisationIsAvailable(List<Directory> activeDirectories) throws IncrementalSynchronisationNotAvailableException
                {
                    for (Directory directory : activeDirectories)
                    {
                        // No events are generated for cacheable directories that are not cached
                        if (isFalse(toBooleanObject(directory.getValue(DirectoryProperties.CACHE_ENABLED))))
                        {
                            throw new IncrementalSynchronisationNotAvailableException("Directory '" + directory.getName() + "' is not cached and so cannot be incrementally synchronised");
                        }
                    }
                }
            {code}
            Where {{com.atlassian.crowd.directory.sync.cache.enabled}} is set to *false*, for the directory's attribute in cwd_directory_attribute. Considering that users and groups for an uncached directory is still cached in the DB anyway, it seems that this check does nothing but to create sync performance issues for customers with delegated directories (it will always fall back to full sync). We should remove this check altogether, or at least, set cache.enabled to *always true*, for *delegated directories*

            h3.Workaround
            Run this query in the Crowd DB:
            {code}
            UPDATE cwd_directory_attribute SET attribute_value = 'true' WHERE attribute_name = 'com.atlassian.crowd.directory.sync.cache.enabled' AND directory_id = <directoryid>;
            {code}
            Where <directoryID> is the directory ID of the delegated directory.

            This will require a Crowd restart

              Unassigned Unassigned
              fsim Foo Sim (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: