Uploaded image for project: 'Crowd Data Center'
  1. Crowd Data Center
  2. CWD-772

Crowd client libraries and caching need further review to improve JIRA performance

    • Icon: Suggestion Suggestion
    • Resolution: Fixed
    • 1.6
    • Caching
    • None
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Due to JIRA's use of OSUser framework for user management, all users, groups and memberships are requested during common actions like Add Project. If Crowd integration is using a large directory, this can cause serious degradation in JIRA performance. The Crowd client libraries may not be caching this data effectively to help rectify this problem.

            [CWD-772] Crowd client libraries and caching need further review to improve JIRA performance

            Hi Aggelos,

            We're planning on investigating it for OpenLDAP, but we don't have a specific timeframe. Please feel free to create an issue for it.

            Unfortunately, sync replication and persistent search are different mechanisms, and it appears that OpenLDAP doesn't expose the Persistent Search control. Although, since there are so many ways to configure OpenLDAP, it's possible that we simply haven't managed to set one of our test instances up to correctly expose it.

            Cheers,
            Dave.

            David O'Flynn [Atlassian] added a comment - Hi Aggelos, We're planning on investigating it for OpenLDAP, but we don't have a specific timeframe. Please feel free to create an issue for it. Unfortunately, sync replication and persistent search are different mechanisms, and it appears that OpenLDAP doesn't expose the Persistent Search control. Although, since there are so many ways to configure OpenLDAP, it's possible that we simply haven't managed to set one of our test instances up to correctly expose it. Cheers, Dave.

            angel added a comment -

            David,

            is the event-based caching mechanism planned or investigated also for OpenLDAP? I can see from the following quote that persistent searches are somehow supported.

            For those who care, OpenLDAP introduced persistent search in its implementation of pull or sync replication (syncrepl) in OpenLDAP 2.2 a couple of years ago. We've considered surfacing it for enterprise use when customers have been curious but none have really pressed for the capability.

            Cheers,
            Aggelos

            angel added a comment - David, is the event-based caching mechanism planned or investigated also for OpenLDAP? I can see from the following quote that persistent searches are somehow supported. For those who care, OpenLDAP introduced persistent search in its implementation of pull or sync replication ( syncrepl ) in OpenLDAP 2.2 a couple of years ago. We've considered surfacing it for enterprise use when customers have been curious but none have really pressed for the capability. Cheers, Aggelos

            Hi folks,

            Crowd 1.6 is now available at the Crowd Download Centre. While we've not made major changes to the client libraries, we have implemented a caching for Active Directory and ApacheDS on the server, which should solve most people's JIRA performance issues.

            I'm going to close this issue. If, after upgrading to 1.6, you're still experiencing problems, please either contact support or open a JIRA ticket specific to your directory or configuration.

            Regards,
            Dave.
            Crowd Product Manager.

            ps: Merry Christmas

            David O'Flynn [Atlassian] added a comment - Hi folks, Crowd 1.6 is now available at the Crowd Download Centre . While we've not made major changes to the client libraries, we have implemented a caching for Active Directory and ApacheDS on the server, which should solve most people's JIRA performance issues. I'm going to close this issue. If, after upgrading to 1.6, you're still experiencing problems, please either contact support or open a JIRA ticket specific to your directory or configuration. Regards, Dave. Crowd Product Manager. ps: Merry Christmas

            We're planning to have Crowd 1.6 out before Christmas.

            David O'Flynn [Atlassian] added a comment - We're planning to have Crowd 1.6 out before Christmas.

            Lanye added a comment -

            The application cache causes delay between JIRA and Crowd which is really inconvenient. It is glad to see that you've put it as the top priority for Crowd 1.6. We'd like to know when is Crowd 1.6 going to be released? Is there any plan for it yet?
            Thanks very much.

            Lanye added a comment - The application cache causes delay between JIRA and Crowd which is really inconvenient. It is glad to see that you've put it as the top priority for Crowd 1.6. We'd like to know when is Crowd 1.6 going to be released? Is there any plan for it yet? Thanks very much.

            We also still get a bit of a hit, even with the new crowd libs. It's not awful, but if you happen to be using JIRA when the cache expires, it's a bit of a wait. FYI, we have ~8000 users.

            I know the design of JIRA necessitates having a copy of all the user data all the time, so I'm not sure what direction you'll need to go. My (unsolicited) suggestion would be to somehow anticipate a cache expiry and have the client update the cache in the background, before the cache actually expires.

            g.

            Graham Bakay added a comment - We also still get a bit of a hit, even with the new crowd libs. It's not awful, but if you happen to be using JIRA when the cache expires, it's a bit of a wait. FYI, we have ~8000 users. I know the design of JIRA necessitates having a copy of all the user data all the time, so I'm not sure what direction you'll need to go. My (unsolicited) suggestion would be to somehow anticipate a cache expiry and have the client update the cache in the background, before the cache actually expires. g.

            I seriously hope you are going to change something in the next release.

            We're working on this as the top priority for Crowd 1.6. We're still in the investigation phase, so I can't tell you exactly what form the solution will take. But one will come

            David O'Flynn [Atlassian] added a comment - I seriously hope you are going to change something in the next release. We're working on this as the top priority for Crowd 1.6. We're still in the investigation phase, so I can't tell you exactly what form the solution will take. But one will come

            We have JIRA 3.12.3, Crowd 1.4 and Confluence 2.8.2

            Our performance is great, we have over 4000 users. We upgraded recently.

            We forgot to put the new crowd client libraries in Confluence and Jira and it was still slow, but once we did that it was much better.

            I don't know why our experience was different.

            Michael Ball added a comment - We have JIRA 3.12.3, Crowd 1.4 and Confluence 2.8.2 Our performance is great, we have over 4000 users. We upgraded recently. We forgot to put the new crowd client libraries in Confluence and Jira and it was still slow, but once we did that it was much better. I don't know why our experience was different.

            I seriously hope you are going to change something in the next release.

            Since we have used the crowd-ehcache.xml file as was advised by the upgrade guide; we are facing slow updates of jira and confluence.
            If we add a user or group to crowd, it now takes like 10 to 30 minutes before I actually see the user or group.
            Very annoying if you want to add the group to a permission scheme directly.

            arno koehler added a comment - I seriously hope you are going to change something in the next release. Since we have used the crowd-ehcache.xml file as was advised by the upgrade guide; we are facing slow updates of jira and confluence. If we add a user or group to crowd, it now takes like 10 to 30 minutes before I actually see the user or group. Very annoying if you want to add the group to a permission scheme directly.

            DonnaA added a comment -

            Hi Eric,

            I have re-opened this because, JIRA's cache will expire on a periodic basis which may cause serious performance hits until the cache has been rebuilt. I think we still have some investigation to do with regard to smarter cache updating. Unfortunately, I cannot provide an estimated fix version for this at this time. I'm hoping to discuss this with the dev team next week to see where we can fit this in the road map or, perhaps, if I should raise a new, more specific issue for this.

            I know you experienced some issues with JIRA integration and Crowd's nested-groups capability in the 1.4 release, which we were able to resolve. I completely understand your hesitancy to upgrade to a new major release as a result of this. Please know the team is working hard to provide a smooth 1.5 release.

            Cheers,
            Donna

            DonnaA added a comment - Hi Eric, I have re-opened this because, JIRA's cache will expire on a periodic basis which may cause serious performance hits until the cache has been rebuilt. I think we still have some investigation to do with regard to smarter cache updating. Unfortunately, I cannot provide an estimated fix version for this at this time. I'm hoping to discuss this with the dev team next week to see where we can fit this in the road map or, perhaps, if I should raise a new, more specific issue for this. I know you experienced some issues with JIRA integration and Crowd's nested-groups capability in the 1.4 release, which we were able to resolve. I completely understand your hesitancy to upgrade to a new major release as a result of this. Please know the team is working hard to provide a smooth 1.5 release. Cheers, Donna

              Unassigned Unassigned
              donna@atlassian.com DonnaA
              Votes:
              13 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved:

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h
                  4h