Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-26067

Bamboo Artifact transfer is failing with : Exception when storing the artifact org.apache.commons.io.FileExistsException: Destination directory already exists

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Low Low
    • 11.0.0
    • 10.2.1
    • Artifacts
    • None

      Issue Summary

      The issue is seen when Artifact having large number of files > 20000 are being transferred from agent and being stored in a shared-home/artifact space which is not on the Bamboo server. 

      Steps to Reproduce

      1) Create a plan which has an Artifact with large number of files ~20000.

      2) The artifact storage should not be on the Bamboo server, in this issue the artifacts were stored on Amazon EFS server.
       
      3) Run the plan.

      Expected Results

      The expectation is that the builds completes and the artifacts are transferred on the shared home folder. 

      Actual Results

      I'll explain this with build logs, agent logs and the Bamboo server logs

      1) We can see on the builds logs that the Artifact transfer failed with http code 500

      simple	24-Mar-2025 11:38:03	Publishing an artifact: XYZ

error	24-Mar-2025 11:57:38	Artifact transfer failed with http code 500.

simple	24-Mar-2025 11:57:38	Finished publishing of artifact Required shared artifact Http Compression On : [XYZ], patterns: [**/*] in 19.59 min

      2) On the Bamboo server we can see that the artifacts started receiving on the <bamboo-shared-home>\artifacts\tmp location at 11:38:23 and first unpacking started by thread http-nio-8086-exec-6

      2025-03-24 11:38:23,266 DEBUG [http-nio-8086-exec-6 url: /agentServer/message] [ArtifactStreams] Unpacking artifact to \\bamboo-shared-home\artifacts\tmp\com.atlassian.bamboo.agent.messaging.ArtifactServlet15644612205549297141

2025-03-24 11:38:23,267 DEBUG [http-nio-8086-exec-6 url: /agentServer/message] [ArtifactStreams] Writing file \\bamboo-shared-home\artifacts\tmp\

      3) For thread http-nio-8086-exec-6 we can see the message related to unpacking at 2025-03-24 11:43:30

      2025-03-24 11:43:30,862 DEBUG [http-nio-8086-exec-6 url: /agentServer/message] [ArtifactStreams] Artifact unpacked to \\bamboo-shared-home\artifacts\tmp\com.atlassian.bamboo.agent.messaging.ArtifactServlet15644612205549297141, packing took 306s, unpacking took 307s

      4) Then after 16 mins we can see the below message where it took 16 mins to transfer the files from /tmp directory to the Artifact location which is very high and point to the slowness of the disk on which the Artifacts are stored. 

      2025-03-24 11:59:35,586 WARN [http-nio-8086-exec-6 url: /agentServer/message] [ArtifactServlet] Artifact processing was longer by 966s than artifact publish, looks like your server is under load or the disk holding the artifact directory is too slow. Deserialisation itself took 307s

2025-03-24 11:59:35,586 INFO [http-nio-8086-exec-6 url: /agentServer/message] [ArtifactServlet] Artifact [XYZ] for ABC-XYZ-BUILD-11 is now available on the server

      6) The second artifact unpacking started at 2025-03-24 11:48:30 by the thread http-nio-8086-exec-7 

      2025-03-24 11:48:30,564 DEBUG [http-nio-8086-exec-7 url: /agentServer/message] [ArtifactStreams] Unpacking artifact to \\bamboo-shared-home\artifacts\tmp\com.atlassian.bamboo.agent.messaging.ArtifactServlet14943093112794594126

      6) The Artifact unpacked at 11:54:34

      2025-03-24 11:54:34,370 DEBUG [http-nio-8086-exec-7 url: /agentServer/message] [ArtifactStreams] Artifact unpacked to \\bamboo-shared-home\artifacts\tmp\com.atlassian.bamboo.agent.messaging.ArtifactServlet14943093112794594126, packing took 362s, unpacking took 363s

      7) Strange thing is both the threads threads http-nio-8086-exec-7 and http-nio-8086-exec-6 are copying exactly the same thing, see below

      http-nio-8086-exec-6
      2025-03-24 11:43:30,782 DEBUG [http-nio-8086-exec-6 url: /agentServer/message] [ArtifactStreams] Writing file \\bamboo-shared-home\artifacts\tmp\com.atlassian.bamboo.agent.messaging.ArtifactServlet15644612205549297141\xmlpull\xmlpull\1.1.3.1\xmlpull-1.1.3.1.jar
      http-nio-8086-exec-7
      2025-03-24 11:54:34,297 DEBUG [http-nio-8086-exec-7 url: /agentServer/message] [ArtifactStreams] Writing file \\bamboo-shared-home\artifacts\tmp\com.atlassian.bamboo.agent.messaging.ArtifactServlet14943093112794594126\xmlpull\xmlpull\1.1.3.1\xmlpull-1.1.3.1.jar

      7) Then we see the below error, this probably is because http-nio-8086-exec-6 was already copying the files and the new thread http-nio-8086-exec-7 started copying to the same location. 

      2025-03-24 11:57:38,179 ERROR [http-nio-8086-exec-7 url: /agentServer/message] [ArtifactServlet] Exception when storing the artifact
org.apache.commons.io.FileExistsException: Destination directory already exists: '\\bamboo-shared-home\artifacts\plan-40304650\shared\build-00011\XYZ'
	at com.atlassian.bamboo.util.BambooFileUtils.moveDirectory(BambooFileUtils.java:368) ~[atlassian-bamboo-core-10.2.2.jar:?]

      From the agent logs we can see below information

      1) The first transfer started from the agent at 11:38:22

      2025-03-24 11:38:22,645 DEBUG [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [BambooArtifactHttpTransport] Artifact [XYZ] for ABC-XYZ-BUILD-11 token successfully verified

2025-03-24 11:38:22,645 DEBUG [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [BambooArtifactHttpTransport] Started writing the artifact to the output stream.

       
      2) It completed the Artifact transfer at 11:43:29 and during this time as per the earlier comment http-nio-8086-exec-6 was moving the files within the shared home directory and hence no request was sent back to the agent and connection timed out with java.net.SocketTimeoutException  at 11:48:29 after 300 seconds as per the setting from bamboo.jms.broker.uri

      2025-03-24 11:43:29,963 INFO [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [BambooArtifactHttpTransport] Finished writing the artifact to the output stream, time elapsed: 5.122 min

2025-03-24 11:48:29,966 INFO [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [RetryExec] I/O exception (java.net.SocketTimeoutException) caught when processing request to {}-&gt;http://0.1.2.3:8086: Read timed out

2025-03-24 11:48:29,966 INFO [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [RetryExec] Retrying request to {}-&gt;http://0.1.2.3:8086 
      <property name="bamboo.jms.broker.uri">tcp://0.0.0.0:54663?wireFormat.maxInactivityDuration=300000&amp;transport.soWriteTimeout=45000</property>

      3) A retry was scheduled and this is the reason which we see http-nio-8086-exec-7 thread was started

      2025-03-24 11:48:29,966 INFO [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [RetryExec] Retrying request to {}-&gt;http://0.1.2.3:8086
2025-03-24 11:48:29,968 DEBUG [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [BambooArtifactHttpTransport] Started writing the artifact to the output stream.
2025-03-24 11:48:30,546 DEBUG [3-BAM::euw1-devba1 - Agent 1::Agent:Thread-31] [ArtifactStreams] Transferring artifact file: .classpath

      Workaround

      Try zipping the artifact as per https://confluence.atlassian.com/bamkb/suggestion-for-optimising-artifacts-size-storage-1345815578.html.

      The idea is to reduce the amount of files which are sent by a single artifact.

            [BAM-26067] Bamboo Artifact transfer is failing with : Exception when storing the artifact org.apache.commons.io.FileExistsException: Destination directory already exists

            There are no comments yet on this issue.

              851f15845f55 Mateusz Szmal
              4f38d6bf51c0 Shashank Kumar
              Affected customers:
              0 This affects my team
              Watchers:
              1 Start watching this issue

                Created:
                Updated: