Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-16970

Fix absolute paths for artifacts being changed to relative paths

    • Icon: Suggestion Suggestion
    • Resolution: Fixed
    • 5.12.4, 5.13.0.1
    • None
    • None
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      When I set an artifact to /var/lib/docker in a build job's properties, it's changed to a relative path

      Unable to publish artifact [Docker Cache]: the source directory /home/bamboo/bamboo-agent-home/xml-data/build-dir/BB-THING-JOB1/var/lib/docker does not exist.
      

      Never in any application should a path that starts with / be changed to a relative path. It specifically breaks making artifacts from global operating system cached files, and it also breaks reasonable expectations for relative files in a POSIX environment.

      I was able to get the artifact published with a six ../ in a row, by changing the artifact path to: ../../../../../../var/lib/docker:

      Publishing an artifact: Docker Cache
      Finished publishing of artifact Shared artifact: [Docker Cache], pattern: [**/*] anchored at: [../../../../../../var/lib/docker] in 5s
      Finalising the build...
      

      However the problem surfaces later during deployment:

      Preparing to download plan result BE-CORE-35 artifact: Shared artifact: [Docker Cache], pattern: [**/*] anchored at: [../../../../../../var/lib/docker]
      Unable to download artifact Shared artifact: [Docker Cache], pattern: [**/*] anchored at: [../../../../../../var/lib/docker]
      

      Because of this bug, I don't see a way to use artifacts that are outside of the build directory unless they're manually moved inside it.

      Workaround

      • create a symbolic link to an external path inside your build directory.

            [BAM-16970] Fix absolute paths for artifacts being changed to relative paths

            I'm sorry but after looking into our logs again, it's a problem of the shell task (when ran from an absolute path) and not the artifacts definitions.

            06-Jul-2017 08:49:36	Error occurred while running Task 'Stop containers' of type com.atlassian.bamboo.plugins.scripttask:task.builder.script.
            06-Jul-2017 08:49:36	com.atlassian.bamboo.task.TaskException: Failed to execute task
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.plugins.shell.task.AbstractShellBuildTask.execute(AbstractShellBuildTask.java:71)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.plugins.script.task.ScriptBuildTask.execute(ScriptBuildTask.java:74)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.task.TaskExecutorImpl.lambda$executeTasks$3(TaskExecutorImpl.java:317)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.task.TaskExecutorImpl.executeTaskWithPrePostActions(TaskExecutorImpl.java:246)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:317)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:122)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:74)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:216)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl$1.call(BuildAgentControllerImpl.java:139)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl$1.call(BuildAgentControllerImpl.java:130)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:221)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:129)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:138)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:51)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52)
            06-Jul-2017 08:49:36	        at java.lang.Thread.run(Thread.java:745)
            06-Jul-2017 08:49:36	Caused by: java.lang.IllegalArgumentException: The working directory /apps/bamboo/xml-data/build-dir/AP-RELBUGFIX1-JOB1/apps/bamboo/xml-data/build-dir/AP-RELBUGFIX1-JOB1/project-release/bin/../../tmp/git must exist before executing the process
            06-Jul-2017 08:49:36	        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.process.ProcessServiceImpl.createExternalProcess(ProcessServiceImpl.java:158)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.process.ProcessServiceImpl.executeExternalProcess(ProcessServiceImpl.java:207)
            06-Jul-2017 08:49:36	        at com.atlassian.bamboo.plugins.shell.task.AbstractShellBuildTask.execute(AbstractShellBuildTask.java:51)
            06-Jul-2017 08:49:36	        ... 17 more

            The shell cwd is defined via a Bamboo var ```${bamboo.ourproject.SOME_PATH}```. Note that the absolute path still falls within the Bamboo working directory, it's just an abs path since it gets generated by a bash script.
             

            Deleted Account (Inactive) added a comment - I'm sorry but after looking into our logs again, it's a problem of the shell task (when ran from an absolute path) and not the artifacts definitions. 06-Jul-2017 08:49:36 Error occurred while running Task 'Stop containers' of type com.atlassian.bamboo.plugins.scripttask:task.builder.script. 06-Jul-2017 08:49:36 com.atlassian.bamboo.task.TaskException: Failed to execute task 06-Jul-2017 08:49:36         at com.atlassian.bamboo.plugins.shell.task.AbstractShellBuildTask.execute(AbstractShellBuildTask.java:71) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.plugins.script.task.ScriptBuildTask.execute(ScriptBuildTask.java:74) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.task.TaskExecutorImpl.lambda$executeTasks$3(TaskExecutorImpl.java:317) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.task.TaskExecutorImpl.executeTaskWithPrePostActions(TaskExecutorImpl.java:246) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:317) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:122) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:74) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:216) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl$1.call(BuildAgentControllerImpl.java:139) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl$1.call(BuildAgentControllerImpl.java:130) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:221) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:129) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:138) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:51) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) 06-Jul-2017 08:49:36         at java.lang. Thread .run( Thread .java:745) 06-Jul-2017 08:49:36 Caused by: java.lang.IllegalArgumentException: The working directory /apps/bamboo/xml-data/build-dir/AP-RELBUGFIX1-JOB1/apps/bamboo/xml-data/build-dir/AP-RELBUGFIX1-JOB1/project-release/bin/../../tmp/git must exist before executing the process 06-Jul-2017 08:49:36         at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.process.ProcessServiceImpl.createExternalProcess(ProcessServiceImpl.java:158) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.process.ProcessServiceImpl.executeExternalProcess(ProcessServiceImpl.java:207) 06-Jul-2017 08:49:36         at com.atlassian.bamboo.plugins.shell.task.AbstractShellBuildTask.execute(AbstractShellBuildTask.java:51) 06-Jul-2017 08:49:36         ... 17 more The shell cwd is defined via a Bamboo var ```${bamboo.ourproject.SOME_PATH}```. Note that the absolute path still falls within the Bamboo working directory, it's just an abs path since it gets generated by a bash script.  

            I just ran into this issue with Bamboo version 5.15.3 build 51515. Can you update the affected versions field (and add a regression test perhaps)?

            Deleted Account (Inactive) added a comment - I just ran into this issue with Bamboo version 5.15.3 build 51515. Can you update the affected versions field (and add a regression test perhaps)?

            Thanks, I think I have already fixed it, I changed all my artifact definitions to remove the prefixed / and rebuilt, now the deployment downloads are happy again.

            If anyone else has these sort of problems, look in the build artifacts, for missing artifacts, that was my clue that something was wrong.

            mark_gillespie added a comment - Thanks, I think I have already fixed it, I changed all my artifact definitions to remove the prefixed / and rebuilt, now the deployment downloads are happy again. If anyone else has these sort of problems, look in the build artifacts, for missing artifacts, that was my clue that something was wrong.

            Przemek Bruski added a comment - - edited

            Edit your bamboo.cfg.xml, change the build number to 51313. Then restart your Bamboo with an extra system variable -DUpgradeTask51314ConvertPathsInDownloadTasks=true .

            Przemek Bruski added a comment - - edited Edit your bamboo.cfg.xml, change the build number to 51313. Then restart your Bamboo with an extra system variable -DUpgradeTask51314ConvertPathsInDownloadTasks=true .

            5.12.1 worked fine.
            5.13.1 is broken (or behaviour has changed)

            mark_gillespie added a comment - 5.12.1 worked fine. 5.13.1 is broken (or behaviour has changed)

            Przemek Bruski added a comment - - edited

            Which version are you on and from which version did you upgrade?

            Przemek Bruski added a comment - - edited Which version are you on and from which version did you upgrade?

            Assuming this is the fix that broke all our build plans

            We have an artifact (a signing key) in the root directory of the repository. It was referenced as /signing.key

            It can no longer find this artifact since upgrading, and the deployments now fail at the download artifact stage.

            mark_gillespie added a comment - Assuming this is the fix that broke all our build plans We have an artifact (a signing key) in the root directory of the repository. It was referenced as /signing.key It can no longer find this artifact since upgrading, and the deployments now fail at the download artifact stage.

            nhooey added a comment -

            So artifact copying doesn't preserve symbolic links, either? Great.

            nhooey added a comment - So artifact copying doesn't preserve symbolic links, either? Great.

            Workaround: create a symbolic link to an external path inside your build directory.

            Marcin Gardias added a comment - Workaround: create a symbolic link to an external path inside your build directory.

              pbruski Przemek Bruski
              025f9d74fb58 nhooey
              Votes:
              2 Vote for this issue
              Watchers:
              14 Start watching this issue

                Created:
                Updated:
                Resolved: