• 6
    • 3
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      The current build expiry system allows you to clean old (and sometimes large) log files from your Bamboo server, but the settings do not affect remote hosts. We should introduce an optional setting, such as a checkbox, for the settings to include remote hosts in the expiry enforcement to allow them to clear out old build data on a regular basis.

          Form Name

            [BAM-13679] Optional build expiry for remote agents

            Charlie.Gunzelman added a comment - - edited

            I solved the issue in a very basic way. Each script below is executed as SSH to localhost as jobs in a Bamboo build plan that gets executed every night.

             
            Prerequisites:
            Generate an SSH key, added the private half to the build plan. The public half of the key was added to the bamboo user’s authorized keys file on each bamboo agent. This should never have to be done again.

            Logic:

            List out the build cache files and folders:

            echo "Deleting the following build cache folders:"
            find /home/bamboo/bamboo-agent-home/bin/ -name plan* -maxdepth 1}}
            cd /home/bamboo/bamboo-agent-home/xml-data/
            find build-dir/ -name C* -not -path "build-dir/repositoryData/*" -maxdepth 1
            find build-dir/ -name T* -not -path "build-dir/repositoryData/*" -maxdepth 1
            find build-dir/_git-repositories-cache -maxdepth 1

            The above code lists out the folders that are deleted in the second half of the plan. I use -maxdepth 1 so only the target folders are listed, not the files and folders contained in them. This reduces the log file size and makes it easier to read. The last 4 lines are a bit interesting because we are only deleting folders that are not repositoryData which probably isn’t necessary, but the folder is relatively small, so I don’t see the need to delete it.

             

            Delete the build cache files and folders:

            echo "Deleting...."
            rm -rf /home/bamboo/bamboo-agent-home/bin/plan*}}
            rm -rf /home/bamboo/bamboo-agent-home/xml-data/build-dir/C*
            rm -rf /home/bamboo/bamboo-agent-home/xml-data/build-dir/T*
            rm -rf /home/bamboo/bamboo-agent-home/xml-data/build-dir/_git-repositories-cache
            echo "Delete finished"

             

             

            Making use of echo " " helps make the job logs easier to read because (by default) bamboo (or bash shell) doesn’t echo the commands as it runs them. There’s probably a bash function to do something like echo on but I don’t feel like messing with that today.

            Screenshot: imgur /a/z1KTcfL

            This has got to be THE WORST markdown editor I've ever used. Had to edit this at least 5 times because it kept adding random curly braces everywhere to my formatted text. {{ }}

            Charlie.Gunzelman added a comment - - edited I solved the issue in a very basic way. Each script below is executed as SSH to localhost as jobs in a Bamboo build plan that gets executed every night.   Prerequisites: Generate an SSH key, added the private half to the build plan. The public half of the key was added to the bamboo  user’s authorized keys file on each bamboo agent. This should never have to be done again. Logic: List out the build cache files and folders: echo "Deleting the following build cache folders:" find /home/bamboo/bamboo-agent-home/bin/ -name plan* -maxdepth 1 }} cd /home/bamboo/bamboo-agent-home/xml-data/ find build-dir/ -name C* -not -path "build-dir/repositoryData/*" -maxdepth 1 find build-dir/ -name T* -not -path "build-dir/repositoryData/*" -maxdepth 1 find build-dir/_git-repositories-cache -maxdepth 1 The above code lists out the folders that are deleted in the second half of the plan. I use - maxdepth 1 so only the target folders are listed, not the files and folders contained in them. This reduces the log file size and makes it easier to read. The last 4 lines are a bit interesting because we are only deleting folders that are not repositoryData  which probably isn’t necessary, but the folder is relatively small, so I don’t see the need to delete it.   Delete the build cache files and folders: echo "Deleting...." rm -rf /home/bamboo/bamboo-agent-home/bin/plan*}} rm -rf /home/bamboo/bamboo-agent-home/xml-data/build-dir/C* rm -rf /home/bamboo/bamboo-agent-home/xml-data/build-dir/T* rm -rf /home/bamboo/bamboo-agent-home/xml-data/build-dir/_git-repositories-cache echo "Delete finished"     Making use of  echo " "  helps make the job logs easier to read because (by default) bamboo (or bash shell) doesn’t echo the commands as it runs them. There’s probably a bash function to do something like  echo on  but I don’t feel like messing with that today. Screenshot: imgur /a/z1KTcfL This has got to be THE WORST markdown editor I've ever used. Had to edit this at least 5 times because it kept adding random curly braces everywhere to my formatted text. {{ }}

            CASG Representative added a comment - - edited

            Will like to see this option available for cache cleanup on the remote agents.

            Umar

            CASG Representative added a comment - - edited Will like to see this option available for cache cleanup on the remote agents. Umar

            gwhite added a comment -

            Even a canned script for which you can set an age, either installed by default or suggested after installation, is IMO far better than "Hmm, where did all my disk go on this remote Bamboo agent box ? " and the only fix is Stackoverflow and cross-your-fingers.

             

             

            gwhite added a comment - Even a canned script for which you can set an age, either installed by default or suggested after installation, is IMO far better than "Hmm, where did all my disk go on this remote Bamboo agent box ? " and the only fix is Stackoverflow and cross-your-fingers.    

            Kevin Webb added a comment -

            Fairly outrageous this hasn't been added yet, or at least commented on why it hasn't been added as a feature.  Other than there are only 6 votes in 6 years.  That may have something to do with it.  Still crazy this is not already existing functionality. 

            Kevin Webb added a comment - Fairly outrageous this hasn't been added yet, or at least commented on why it hasn't been added as a feature.  Other than there are only 6 votes in 6 years.  That may have something to do with it.  Still crazy this is not already existing functionality. 

            This functionality would be incredibly useful for my team, where we have node_modules directories causing every single build-dir to be ~1GB. Bamboo reuses directories for each git branch, but as we create new branches we end up with hundreds of copies of these dependencies. Disk space can run out pretty quickly on our remote agents, and it's annoying that there is no automated way to clean this up.

            Hank Brekke added a comment - This functionality would be incredibly useful for my team, where we have node_modules directories causing every single build-dir to be ~1GB. Bamboo reuses directories for each git branch, but as we create new branches we end up with hundreds of copies of these dependencies. Disk space can run out pretty quickly on our remote agents, and it's annoying that there is no automated way to clean this up.

            Why is this resolved? This is a feature I would like to see. I'd like builds on agents to adhere to the expiry policy defined in Bamboo and not "unused" build directories as the only criteria.

            Eliza Ralph added a comment - Why is this resolved? This is a feature I would like to see. I'd like builds on agents to adhere to the expiry policy defined in Bamboo and not "unused" build directories as the only criteria.

            Hi Adam,

            We probably won't add this to the expiry options but I think we should remove unused build directories on remote agents periodically.

            Thanks
            James

            James Dumay added a comment - Hi Adam, We probably won't add this to the expiry options but I think we should remove unused build directories on remote agents periodically. Thanks James

              Unassigned Unassigned
              alaskowski Adam Laskowski (Inactive)
              Votes:
              37 Vote for this issue
              Watchers:
              24 Start watching this issue

                Created:
                Updated: