Uploaded image for project: 'Bitbucket Cloud'
  1. Bitbucket Cloud
  2. BCLOUD-21785

Pipelines build not triggered correctly when executing large number of pipes in a single build

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Low Low
    • Pipelines - Pipes
    • None

      Issue Summary

      If bitbucket-pipelines.yml is configured to run a large number of pipes in a single build, Pipelines will exert the following behaviours:

      1. Received "An unidentified error occurred" while running Pipelines through UI
      2. Multiple builds of the same commit triggered after a single push to the branch

      This greatly impacts user who has a large number of AWS lambda function and relies on Pipe to deploy it.

      Steps to Reproduce

      Duplicate the following pipes 190 times in the bitbucket-pipelines.yml config:

      pipelines:
        default:
          - step:
              script:
                - pipe: docker://bitbucketpipelines/aws-lambda-deploy:0.5.0
                   variables:
                       AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID_STAGE
                       AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY_STAGE
                       AWS_DEFAULT_REGION: $AWS_REGION_STAGE
                       FUNCTION_NAME: my_function_name
                       COMMAND: "update"
                - pipe: docker://bitbucketpipelines/aws-lambda-deploy:0.5.0
                   variables:
                       AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID_STAGE
                       AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY_STAGE
                       AWS_DEFAULT_REGION: $AWS_REGION_STAGE
                       FUNCTION_NAME: my_function_name
                       COMMAND: "update"
      .....
      .....
      

      Ideally, the "FUNCTION_NAME" will be different in different pipes, but we're able to reproduce with the same config as above

      Expected Results

      • Users able to run build through UI without error
      • Pipelines only trigger a single build after a push is performed.

      Actual Results

      1. Received "An unidentified error occurred" while running Pipelines through UI
      2. Multiple builds triggered on the same commit, after a single push to the branch

      Workaround

      Two workarounds:

      1. Reduce the number of Pipes within a single build
        • If you need to trigger more than 190 pipes, you can configure the remaining Pipes to run in subsequence build with Pipelines custom step and trigger it with Trigger Pipelines Pipe
          Example:
          pipelines:
            default:
              - step:
                  script:
                    - pipe: docker://bitbucketpipelines/aws-lambda-deploy:0.5.0
                       variables:
          ......
              - step:
                  name: Trigger subsequent pipeline with custom step
                  script:
                    - pipe: atlassian/trigger-pipeline:5.0.0
                       variables:
                            BITBUCKET_USERNAME: '<string>'
                            BITBUCKET_APP_PASSWORD: '<string>'
                            REPOSITORY: '<string>'
                            ACCOUNT: '<string>' # Optional
                            REF_TYPE: 'branch' # Optional
                            REF_NAME: 'stage' # Optional
                            CUSTOM_PIPELINE_NAME: 'SubsequentStepswithpipes' 
                            # PIPELINE_VARIABLES: '<json>' # Optional
                            # WAIT: '<boolean>' # Optional
                            # WAIT_MAX_TIMEOUT: '<string>' # Optional
                            # DEBUG: '<boolean>' # Optional
              custom : 
                SubsequentStepswithpipes : 
                 - step:
                     script : 
                     - pipe: atlassian/aws-ecr-push-image:1.1.1
          .....
          
      2. If it's an Atlassian Pipes(pipe: atlassian/****), you can replace the Pipe name with
        pipe: docker://bitbucketpipelines/*****
        

        For example: pipe: docker://bitbucketpipelines/aws-lambda-deploy:0.5.0

              Unassigned Unassigned
              rchia Ronald C.
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: