Uploaded image for project: 'Crowd Data Center'
  1. Crowd Data Center
  2. CWD-848

Using the Importers (CSV, Atlassian Importer, LDAP) will be slow with large datasets.

    XMLWordPrintable

Details

    Description

      Currently the importers, particularly when they are using Internal Directories are inefficient in their use of Memory and also their use of Hibernate.

      We need to tackle this in about 3 ways:

      1. Look into the memory issues, when dealing with large lists of users/groups/memberships
      2. Add new methods to the RemoteDirectory and DirectoryManager specifically around bulk loading Groups, Principals and Memberships.
      3. For the internal directory use a stateless session for the bulk adding, skipping the 1st level cache.

      Attachments

        Issue Links

          causes

          Suggestion - CWD-1175 Please add a note about this bug on our import docs

          • Closed
          is related to

          Suggestion - CWD-1212 Improving memory usage when importing large datasets

          • Gathering Interest
          relates to

          Suggestion - CWD-831 Importing of larger users bases (cvs, atlassian, jive) will cause the Crowd server to run out of memory

          • Closed

          Activity

            People

              justin@atlassian.com Justin Koke
              justin@atlassian.com Justin Koke
              Votes:
              4 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: