Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-2423

Mechanism to prevent builds that use a shared resource from running concurrently

    • 3
    • 11
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      It is often the case that there are multiple build plans that make use of a shared resource in the testing environment, and that running these concurrently causes problems.

      Bamboo needs a mechanism, such as a special kind of build requirement, that prevents such build plans from being run concurrently.

            [BAM-2423] Mechanism to prevent builds that use a shared resource from running concurrently

            Atlassian Update

            Hi everyone,

            Thank you for your interest in this issue.

            While this suggestion has gathered significant interest, we're unable to implement all of the excellent suggestions you make. We don't plan to work on this for the foreseeable future.

            We'll typically review this request in about 6 months time, at which point we’ll consider whether we need to alter its status.

            We understand this decision will be disappointing to everyone who voted for this issue. While we believe this suggestion would improve the product, after careful review of the most pressing needs of our customers, we've decided to prioritise other areas of the Bamboo roadmap, including:

            1. Robustness of Plan Branches
            2. Performance and stability improvements
            3. Providing building blocks for High Availability and Disaster Recovery solutions
            4. Improving permission system
            5. Allowing per-project allocation of resources
            6. Improving Bitbucket Server and Jira integrations

            We hope that you appreciate our candid and transparent communication. You can learn more about our approach to highly voted server suggestions here.

            To learn more on how your suggestions are reviewed, see our updated workflow for server feature suggestions.

            Kind regards,
            Bamboo Team

            Krystian Brazulewicz added a comment - Atlassian Update Hi everyone, Thank you for your interest in this issue. While this suggestion has gathered significant interest, we're unable to implement all of the excellent suggestions you make. We don't plan to work on this for the foreseeable future. We'll typically review this request in about 6 months time, at which point we’ll consider whether we need to alter its status. We understand this decision will be disappointing to everyone who voted for this issue. While we believe this suggestion would improve the product, after careful review of the most pressing needs of our customers, we've decided to prioritise other areas of the Bamboo roadmap, including: Robustness of Plan Branches Performance and stability improvements Providing building blocks for High Availability and Disaster Recovery solutions Improving permission system Allowing per-project allocation of resources Improving Bitbucket Server and Jira integrations We hope that you appreciate our candid and transparent communication. You can learn more about our approach to highly voted server suggestions here . To learn more on how your suggestions are reviewed, see our updated workflow for server feature suggestions . Kind regards, Bamboo Team

            pinebit added a comment -

            +1

            pinebit added a comment - +1

            +99999

            +1

            this feature is very important for testing purpose.

            Xavier Fernandes added a comment - +1 this feature is very important for testing purpose.

            Gerald Chu added a comment -

            +1

            Gerald Chu added a comment - +1

            +1

            Alex Bandtock added a comment - +1

            +1

            Joao Leal added a comment -

            +1

            Joao Leal added a comment - +1

            +1

            Okke Hendriks added a comment - +1

            bassvdo added a comment -

            +1

            bassvdo added a comment - +1

            +1

            Why is such an important and voted issue opened from 2008? Should we migrate to TeamCity?

            ondrej_lerch added a comment - Why is such an important and voted issue opened from 2008? Should we migrate to TeamCity?

            A rather clunky workaround documented on stackoverflow http://stackoverflow.com/questions/36538061/bamboo-limit-concurrent-builds-across-branches might help some users in the interim...

            Craig Oliver added a comment - A rather clunky workaround documented on stackoverflow http://stackoverflow.com/questions/36538061/bamboo-limit-concurrent-builds-across-branches might help some users in the interim...

            Seeing another important feature lurking around for a very long time just beeing ignored by Atlassian makes me very sad indeed.
            This one is even in the Top 50 of most voted issues.

            Andreas Küchler added a comment - Seeing another important feature lurking around for a very long time just beeing ignored by Atlassian makes me very sad indeed. This one is even in the Top 50 of most voted issues.

            +1

            Henk Triegel added a comment - +1

            erlendb added a comment -

            +1

            erlendb added a comment - +1

            For the love of god (or deity of your choice)... someone make this happen!

            lukeswindale added a comment - For the love of god (or deity of your choice)... someone make this happen!

            Adam Barna added a comment -

            I wanted to add my voice that this is a very important feature for a mature CI tool to have. Please implement it soon!

            Adam Barna added a comment - I wanted to add my voice that this is a very important feature for a mature CI tool to have. Please implement it soon!

            Judging by Atlassian's [lack of] response, I doubt we'll ever have it, Carl.

            Deleted Account (Inactive) added a comment - Judging by Atlassian's [lack of] response, I doubt we'll ever have it, Carl.

            Surprised that this is still not available in Bamboo.

            Carl Samuelson added a comment - Surprised that this is still not available in Bamboo.

            +1

            We have a set of test builds that hit the same resources heavily. We don't want to run them at the same time. Nothing currently in Bamboo allows us to do this so we're stuck babysitting the builds manually.

            Deleted Account (Inactive) added a comment - +1 We have a set of test builds that hit the same resources heavily. We don't want to run them at the same time. Nothing currently in Bamboo allows us to do this so we're stuck babysitting the builds manually.

            +1
            We really need a "Don't Build Concurrently With" plan option. It's a missing core feature, really
            Shouldn't be hard to implement

            Rune Engseth added a comment - +1 We really need a "Don't Build Concurrently With" plan option. It's a missing core feature, really Shouldn't be hard to implement

            +1 to get this into Bamboo.

            Charles' plugin doesn't work as smoothly with newer versions of Bamboo and so we're getting "stuck" here. This happens frequently:

            Step 1 : Test build prepares environment
            Steps 2-10 : Tests run in parallel against environment

            We have multiple Plans which do these steps. If step 1 is run on another plan while steps 2-10 are running, then that re-initializes the environment and prevents the tests from running properly.

            Dana Lacoste added a comment - +1 to get this into Bamboo. Charles' plugin doesn't work as smoothly with newer versions of Bamboo and so we're getting "stuck" here. This happens frequently: Step 1 : Test build prepares environment Steps 2-10 : Tests run in parallel against environment We have multiple Plans which do these steps. If step 1 is run on another plan while steps 2-10 are running, then that re-initializes the environment and prevents the tests from running properly.

            Charles added a comment -

            I had a quick look into it, but it seemed like getting the plugin onto the marketplace was going to be too many hoops to jump through for such a simple, free plugin. Maybe someday if it proves popular and I have the time.

            Charles added a comment - I had a quick look into it, but it seemed like getting the plugin onto the marketplace was going to be too many hoops to jump through for such a simple, free plugin. Maybe someday if it proves popular and I have the time.

            Thats great Charles. Are you planning to get this onto the Atlassian Marketplace at all?

            James Dumay added a comment - Thats great Charles. Are you planning to get this onto the Atlassian Marketplace at all?

            Charles added a comment -

            I implemented a simple plugin to solve this problem a while back and have just gotten approval to open source it. Source and documentation can be found here: https://github.com/ampedandwired/bamboo-mutex-plugin.

            This would still be better implemented within Bamboo core I think, but this plugin should do the job until then.

            Charles added a comment - I implemented a simple plugin to solve this problem a while back and have just gotten approval to open source it. Source and documentation can be found here: https://github.com/ampedandwired/bamboo-mutex-plugin . This would still be better implemented within Bamboo core I think, but this plugin should do the job until then.

            I would like to reiterate how important this is to my team. It's been 31 weeks since I posted our wish for this to be implemented, but I don't want to leave the impression that we have a workaround or that everything is getting along fine without this fix. We are no better off here, and the problem is now five times worse because we have so many more plans and builds in process. I don't know what it might take to convince Atlassian to make this happen sooner, but please don't let lack of continued comments lead anyone to believe that things are better.

            Deleted Account (Inactive) added a comment - I would like to reiterate how important this is to my team. It's been 31 weeks since I posted our wish for this to be implemented, but I don't want to leave the impression that we have a workaround or that everything is getting along fine without this fix. We are no better off here, and the problem is now five times worse because we have so many more plans and builds in process. I don't know what it might take to convince Atlassian to make this happen sooner, but please don't let lack of continued comments lead anyone to believe that things are better.

            We also need atomic plan execution. If three different builds on the same agent are concurrently executing, it makes the build completion time a useless statistic, not to mention all of the complications above.

            We would like to be able to avoid the interleaving execution of jobs from diffent plans. It is causing us a number of problems without any workaround at this time.

            Deleted Account (Inactive) added a comment - We also need atomic plan execution. If three different builds on the same agent are concurrently executing, it makes the build completion time a useless statistic, not to mention all of the complications above. We would like to be able to avoid the interleaving execution of jobs from diffent plans. It is causing us a number of problems without any workaround at this time.

            We have the exact same problem: Different plans that use a shared resource, but are otherwise independent, are allowed by Bamboo to execute interleaved/intermixed jobs, on a given agent. This causes problems, and we would like to somehow make sure that those plans are not allowed to interleave the execution of their stages/jobs.

            I vote for a solution

            Erik Underbjerg added a comment - We have the exact same problem: Different plans that use a shared resource, but are otherwise independent, are allowed by Bamboo to execute interleaved/intermixed jobs, on a given agent. This causes problems, and we would like to somehow make sure that those plans are not allowed to interleave the execution of their stages/jobs. I vote for a solution

            AntonA added a comment -

            Thanks for the clarification.

            AntonA added a comment - Thanks for the clarification.

            Hi Anton,

            Sorry for not making this clearer.

            By atomic I mean that the plan should run all stages on a particular agent before any other plan gets to run any stages on that agent. At the moment stages are atomic, it would be nice to allow plans to be atomic also.

            HTH,

            Paudi

            Paudi Moriarty added a comment - Hi Anton, Sorry for not making this clearer. By atomic I mean that the plan should run all stages on a particular agent before any other plan gets to run any stages on that agent. At the moment stages are atomic, it would be nice to allow plans to be atomic also. HTH, Paudi

            AntonA added a comment -

            Hi,

            I am not sure what you mean by "plan execution is not atomic".

            If Concurrent Builds are disabled, then a single plan would never execute concurrently, therefore plan execution is atomic.

            If Concurrent Builds are enabled then the build execution is not atomic.

            Do you mean you would like us to make the Concurrent Builds documentation more prominent?

            Cheers,
            Anton

            AntonA added a comment - Hi, I am not sure what you mean by "plan execution is not atomic". If Concurrent Builds are disabled, then a single plan would never execute concurrently, therefore plan execution is atomic. If Concurrent Builds are enabled then the build execution is not atomic. Do you mean you would like us to make the Concurrent Builds documentation more prominent? Cheers, Anton

            Thanks Anton,

            I've been struggling to have this understood for a while. In the meantime, a note in the docs to make it clear that plan execution is not atomic might prevent further confusion. I certainly assumed it would be.

            Paudi Moriarty added a comment - Thanks Anton, I've been struggling to have this understood for a while. In the meantime, a note in the docs to make it clear that plan execution is not atomic might prevent further confusion. I certainly assumed it would be.

            AntonA added a comment -

            Thanks for the update. I certainly see what you mean.

            AntonA added a comment - Thanks for the update. I certainly see what you mean.

            Dependencies are not a good fit. My plans are not interdependent in the usual sense, they just share a resource and may conflict if they are not executed atomically. Using Bamboo dependencies, I would have to make one plan a parent and another a child. This would cause unnecessary building as the parent plan would trigger the child plan(s).

            I can certainly work around the problem and avoid the conflict by modifying the source code. I already said this was a problem of convenience rather than ability:

            It is convenient to have these plans build the same database but interleaved stage execution prevents this.

            Paudi Moriarty added a comment - Dependencies are not a good fit. My plans are not interdependent in the usual sense, they just share a resource and may conflict if they are not executed atomically. Using Bamboo dependencies, I would have to make one plan a parent and another a child. This would cause unnecessary building as the parent plan would trigger the child plan(s). I can certainly work around the problem and avoid the conflict by modifying the source code. I already said this was a problem of convenience rather than ability: It is convenient to have these plans build the same database but interleaved stage execution prevents this.

            You can't implement the ordering that you need via dependencies?
            Or wire something into your Test fixture to check the resource you're depending on and its state? READY/NOT_READY, etc?

            Rene Medellin [Atlassian] added a comment - You can't implement the ordering that you need via dependencies? Or wire something into your Test fixture to check the resource you're depending on and its state? READY/NOT_READY, etc?

            This could also prevent interleaved execution of stages from different plans.

            I have multiple plans (trunk, branch A, etc) which build a database in stage 1 in preparation for stages 2,3,...

            It is convenient to have these plans build the same database but interleaved stage execution prevents this.

            Would be nice to be able to disallow this.

            Paudi Moriarty added a comment - This could also prevent interleaved execution of stages from different plans. I have multiple plans (trunk, branch A, etc) which build a database in stage 1 in preparation for stages 2,3,... It is convenient to have these plans build the same database but interleaved stage execution prevents this. Would be nice to be able to disallow this.

              Unassigned Unassigned
              ahempel Adrian Hempel [Atlassian]
              Votes:
              159 Vote for this issue
              Watchers:
              84 Start watching this issue

                Created:
                Updated:

                  Estimated:
                  Original Estimate - 80h
                  80h
                  Remaining:
                  Remaining Estimate - 80h
                  80h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified