Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-17403

Clean up build working directory similar to the expiry configuration

    • Icon: Suggestion Suggestion
    • Resolution: Unresolved
    • None
    • Expiry
    • None
    • 1
    • 7
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      It would be nice to have the ability to clean up the build working directory similar to the Expiry configuration. The Expiry tool provides the ability to remove all build results data (including artifacts and build logs), deployment results and release artifacts, and specify the age (days, weeks or months) they must reach before they get removed. However this doesn't cover the build working directory.

      There should be a way to clean up older working directories and not only the current ones. The "Clean working directory after each build" option inside the Job configuration area under the Miscellaneous tab doesn't help, in this case, because the build time increase is not something scalable in a Continuous Integration build.

      For example, if we have a Plan running each hour, the "Clean working directory after each build" option means all the artifacts would need to be downloaded again. It would be good to have a feature that provides such flexibility just like the Global Expiry.

            [BAM-17403] Clean up build working directory similar to the expiry configuration

            Ah yes, nothing like having a build plan with a thousand branches, and then having to go to each different agent to manually delete all of the no-longer existing branch build directories.

            Georgi Yankov added a comment - Ah yes, nothing like having a build plan with a thousand branches, and then having to go to each different agent to manually delete all of the no-longer existing branch build directories.

            That is almost a full year of time spent on this FOR EACH UPGRADE!

            admin admin added a comment - That is almost a full year of time spent on this FOR EACH UPGRADE!

            Why isn't this implemented yet?
            It takes a system admin 30 seconds to implement a cron job to do this; Imagine the time consumed by several million customers each using 30 seconds to do something which SHOULD be automated; approximating 8333 hours spent on each upgrade, assuming the admins understand exactly what to do, and have an automated process, anything else will increase this time by powers of 10.

            This is VERY disrespectful for your customers at large and should already be part of the package.

            admin admin added a comment - Why isn't this implemented yet? It takes a system admin 30 seconds to implement a cron job to do this; Imagine the time consumed by several million customers each using 30 seconds to do something which SHOULD be automated; approximating 8333 hours spent on each upgrade, assuming the admins understand exactly what to do, and have an automated process, anything else will increase this time by powers of 10. This is VERY disrespectful for your customers at large and should already be part of the package.

            Apologies for the scripting errors above, comments did not implement bold/italics as I expected, the script is as follows;
            0 6 * * * find /var/atlassian/bamboo/xml-data/build-dir/* -mtime +2 -exec rm -rf {} \;

            admin admin added a comment - Apologies for the scripting errors above, comments did not implement bold/italics as I expected, the script is as follows; 0 6 * * * find /var/atlassian/bamboo/xml-data/build-dir/* -mtime +2 -exec rm -rf {} \;

            This issue halted our server until we investigated to determine the cause, and develop a solution for it.
            I can understand how this would appear to be in the realm of server maintenance, but it isn't. Simply put; this app (Bamboo) will disable itself when it builds to much (resulting in excess storage used in

            {BAMBOO_HOME}

            \xml-data\build-dir\ ). As the app itself is disabling it's own operation, I request a feature to resolve this in an automated way;

            I am aware that various servers have different utilities, but two (primary) supported distros for this product appear to be the Debian and RHEL, both of which utilize cron;
            So from an administrative side, this requires installation of a cron job;
            _ 0 6 * * * find /var/atlassian/bamboo/xml-data/build-dir/ -mtime +${BUILD_OBJECT_DURATION} -exec rm -rf {} \;_*

            Note that the prefix (job initiation time) is arbitrary.
            If Atlassian installs such a cronjob, they would then only need to instantiate the env_var BUILD_OBJECT_DURATION to be the length of time (in days) that objects should remain in this directory, which can also be implemented as modifiable through the web-gui for easy/quick modification.

            NOTE: This solution does require the machine to be in an operational state during the prefix period, otherwise the removal job is not executed.

            admin admin added a comment - This issue halted our server until we investigated to determine the cause, and develop a solution for it. I can understand how this would appear to be in the realm of server maintenance, but it isn't. Simply put; this app (Bamboo) will disable itself when it builds to much (resulting in excess storage used in {BAMBOO_HOME} \xml-data\build-dir\ ). As the app itself is disabling it's own operation, I request a feature to resolve this in an automated way; I am aware that various servers have different utilities, but two (primary) supported distros for this product appear to be the Debian and RHEL, both of which utilize cron; So from an administrative side, this requires installation of a cron job; _ 0 6 * * * find /var/atlassian/bamboo/xml-data/build-dir/ -mtime +${BUILD_OBJECT_DURATION} -exec rm -rf {} \;_* Note that the prefix (job initiation time) is arbitrary. If Atlassian installs such a cronjob, they would then only need to instantiate the env_var BUILD_OBJECT_DURATION to be the length of time (in days) that objects should remain in this directory, which can also be implemented as modifiable through the web-gui for easy/quick modification. NOTE: This solution does require the machine to be in an operational state during the prefix period, otherwise the removal job is not executed.

            Dave added a comment -

            A useful workflow would be to have a Bamboo option to make build agents automatically delete the working directory for a branch plan when that branch plan is removed/deleted from the build plan.

            Dave added a comment - A useful workflow would be to have a Bamboo option to make build agents automatically delete the working directory for a branch plan when that branch plan is removed/deleted from the build plan.

            I completely agree with Chad Yates, Most build server installations dedicate some machine (VM or physical) to performing these builds. The Agent should at least be able to help maintain the systems in the state that I deployed them, and not be modifying that state over time by not cleaning up and restoring the build machine to useable.

            I'll extend this to say that the Bamboo system should let me do effective fleet management of these machines in general (Like, install a new package/run a script across many machines of a type to "Add" a capability – (see https://www.saltstack.com/ for reference).

            Jedidiah Bartlett added a comment - I completely agree with Chad Yates, Most build server installations dedicate some machine (VM or physical) to performing these builds. The Agent should at least be able to help maintain the systems in the state that I deployed them, and not be modifying that state over time by not cleaning up and restoring the build machine to useable. I'll extend this to say that the Bamboo system should let me do effective fleet management of these machines in general (Like, install a new package/run a script across many machines of a type to "Add" a capability – (see https://www.saltstack.com/ for reference).

            Chad Yates added a comment -

            In my opinion, this is not a nice to have feature but is a glaring omission, and should be considered a usability bug at the least.  I see where a similar issue was requested, and resolved as will-not-implement.

            Deploying custom cron-jobs and scheduled windows tasks is a work-a-round, but Bamboo should be providing rules to maintain a hands-free agent pool.  While using the clean before/after each build options are useful for ensuring a "clean environment" for builds that require such a policy, they don't work well from an energy/time standpoint for the ideal quick turn-around and immediate feedback of the first level of continuous integration builds.

            I would like to see an option to expire build working directories, as well as options to remove them when a branch/branch-plan no longer exists.

            Chad Yates added a comment - In my opinion, this is not a nice to have feature but is a glaring omission, and should be considered a usability bug at the least.  I see where a similar issue was requested, and resolved as will-not-implement. Deploying custom cron-jobs and scheduled windows tasks is a work-a-round, but Bamboo should be providing rules to maintain a hands-free agent pool.  While using the clean before/after each build options are useful for ensuring a "clean environment" for builds that require such a policy, they don't work well from an energy/time standpoint for the ideal quick turn-around and immediate feedback of the first level of continuous integration builds. I would like to see an option to expire build working directories, as well as options to remove them when a branch/branch-plan no longer exists.

              Unassigned Unassigned
              brosa Bruno Rosa
              Votes:
              33 Vote for this issue
              Watchers:
              19 Start watching this issue

                Created:
                Updated: