Uploaded image for project: 'Crowd Data Center'
  1. Crowd Data Center
  2. CWD-2809

Use a single query for cross-directory searches when all directories are local

XMLWordPrintable

    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      When querying across directories, Crowd currently queries them individually and then aggregates in-memory.

      When all directories are stored locally, either as local directories or cached remote, aggregating across directories can happen directly in the database for a performance increase.

            [CWD-2809] Use a single query for cross-directory searches when all directories are local

            Mark Lassau (Inactive) added a comment -

            Aggregation in this case means two different things: sorting and removing shadowed users.
            Both of these have problems if you try to do them directly in the DB.
            Removing shadowed users I believe is too complex for direct DB manipulation and sorting could produce different results to today based on the collation.

            Furthermore some DAO implementations (JIRA) include in memory caches, so directly calling the DB would lead to a massive decrease in performance.

            Instead there are other inefficiencies that we could explore:

            • ApplicationServiceGeneric lowercases the usernames for duplicate detection
              We already store the lowercase name in the DB but we don't expose it through the User object
            • ApplicationServiceGeneric sorts Users even when there are is no "maxResults" in the query
              Sorting is only needed when paging.
              An ALL_RESULTS query is going to be big and is most in need of optimisation.
            • ApplicationServiceGeneric sorts all Users
              We only need to sort up to the end of the current page.
              Users that don't fall into that subset can be discarded without full sorting

            Mark Lassau (Inactive) added a comment - Aggregation in this case means two different things: sorting and removing shadowed users. Both of these have problems if you try to do them directly in the DB. Removing shadowed users I believe is too complex for direct DB manipulation and sorting could produce different results to today based on the collation. Furthermore some DAO implementations (JIRA) include in memory caches, so directly calling the DB would lead to a massive decrease in performance. Instead there are other inefficiencies that we could explore: ApplicationServiceGeneric lowercases the usernames for duplicate detection We already store the lowercase name in the DB but we don't expose it through the User object ApplicationServiceGeneric sorts Users even when there are is no "maxResults" in the query Sorting is only needed when paging. An ALL_RESULTS query is going to be big and is most in need of optimisation. ApplicationServiceGeneric sorts all Users We only need to sort up to the end of the current page. Users that don't fall into that subset can be discarded without full sorting

              Unassigned Unassigned
              jwalton joe
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: