Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-2989

Artifact collections with large numbers of small files take forever to copy

XMLWordPrintable

    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      It appears from looking at log files that each individual file in an artifact collection is transmitted as a single JMS message. This isn't particularly great for a lot of our plans, where we copy over a hierarchy of JUnit HTML and Clover HTML reports. They can take many minutes to copy the artifacts (where artifact copying takes far longer than the actual build itself).

      Would it be possible to batch up artifact copying in some way to reduce the length of time it takes to copy large numbers of small files?

            [BAM-2989] Artifact collections with large numbers of small files take forever to copy

            edwin added a comment -

            Kyle,

            We have done a lot of work in the current 2.2. release around improving artifact transfer speeds from remote agents. This should solve the issue you are seeing.

            Please do re-open this issue if artifact transfer is still an issue for you after release 2.2. (which is not too far away).

            Cheers,
            Edwin

            edwin added a comment - Kyle, We have done a lot of work in the current 2.2. release around improving artifact transfer speeds from remote agents. This should solve the issue you are seeing. Please do re-open this issue if artifact transfer is still an issue for you after release 2.2. (which is not too far away). Cheers, Edwin

            Kirk Wylie added a comment -

            By the way, at the moment we have one plan that from a remote agent takes over 30 minutes just to copy artifacts. Admittedly, it's 3.2GB of them, but the disks aren't currently loaded and neither is the network on the machine.

            During this period the entire Bamboo system totally freezes up and we can't kick off anything new at all.

            Kirk Wylie added a comment - By the way, at the moment we have one plan that from a remote agent takes over 30 minutes just to copy artifacts. Admittedly, it's 3.2GB of them, but the disks aren't currently loaded and neither is the network on the machine. During this period the entire Bamboo system totally freezes up and we can't kick off anything new at all.

              Unassigned Unassigned
              aca233f59075 Kirk Wylie
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved:

                  Estimated:
                  Original Estimate - 40h
                  40h
                  Remaining:
                  Remaining Estimate - 40h
                  40h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified