• Icon: Suggestion Suggestion
    • Resolution: Unresolved
    • None
    • Builds
    • None
    • 2
    • 3
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Our build process writes status to STDOUT, such as the current script that is executing. Sometimes one of our scripts hangs. We can't tell which script is hanging via the Bamboo UI because the Activity tab appears to buffer output. The last line written to STDOUT is critical. We'd rather not try to outsmart Bamboo by writing a lot of bogus lines to STDOUT.

            [BAM-8029] STDOUT and STDERR buffering is very annoying

            Steven Roodhorst added a comment - - edited

            Managed to fix this problem by using:

            unbuffer <command> |& tee -a unittest.log
            

            Will probably also work without 'tee'.

             

            Steven Roodhorst added a comment - - edited Managed to fix this problem by using: unbuffer <command> |& tee -a unittest.log Will probably also work without 'tee'.  

            Looking at the agent logs, when I get cases of no logs being seen in the bamboo UI, I see agent logs like this:

            2020-09-07 12:39:39,843 INFO [68:StdOutHandler [/bin/sh]] [RemoteBuildLogger] Suspending live log transmission
            

            Ian Norton added a comment - Looking at the agent logs, when I get cases of no logs being seen in the bamboo UI, I see agent logs like this: 2020-09-07 12:39:39,843 INFO [68:StdOutHandler [/bin/sh]] [RemoteBuildLogger] Suspending live log transmission

            Ian Norton added a comment -

            I've been fighting this again recently. At first I thought my application and build were not flushing stdout.  Sometimes I'd get incremental logs, other times I'd get nothing at all until the job completes.

            On some of these builds the logs are over 40mb of text. So I can't believe this is shell/pipe buffering.

            I've seen instances where I get bamboo's early internal messages upto and including:

            simple	24-Aug-2020 07:29:54	Finished task 'Checkout Default Repository' with result: Success
            simple	24-Aug-2020 07:29:54	Running pre-build action: Artifactory Release Staging
            simple	24-Aug-2020 07:29:54	Running pre-build action: VCS Version Collector
            

            Then nothing at all until the job is cancelled or ends naturally. Not even the next internal line from bamboo which is usually like:

            simple	24-Aug-2020 07:29:54	Starting task 'build' of type 'com.atlassian.bamboo.plugins.scripttask:task.builder.script'
            

            Ian Norton added a comment - I've been fighting this again recently. At first I thought my application and build were not flushing stdout.  Sometimes I'd get incremental logs, other times I'd get nothing at all until the job completes. On some of these builds the logs are over 40mb of text. So I can't believe this is shell/pipe buffering. I've seen instances where I get bamboo's early internal messages upto and including: simple 24-Aug-2020 07:29:54 Finished task 'Checkout Default Repository' with result: Success simple 24-Aug-2020 07:29:54 Running pre-build action: Artifactory Release Staging simple 24-Aug-2020 07:29:54 Running pre-build action: VCS Version Collector Then nothing at all until the job is cancelled or ends naturally. Not even the next internal line from bamboo which is usually like: simple 24-Aug-2020 07:29:54 Starting task 'build' of type 'com.atlassian.bamboo.plugins.scripttask:task.builder.script'

            Ian Norton added a comment -

            For anyone else coming along after years of this, the above stdbuf trick only works in some cases.  

            stdbuf uses LD_PRELOAD so if your application is statically linked this won't have any impact.

            I would though say this is a bad user experience in bamboo.  Other systems cope with this.

            Ian Norton added a comment - For anyone else coming along after years of this, the above stdbuf trick only works in some cases.   stdbuf uses LD_PRELOAD so if your application is statically linked this won't have any impact. I would though say this is a bad user experience in bamboo.  Other systems cope with this.

            For future people that run into this:  we though we were having trouble with bamboo buffering the output of our jobs, but it wasn't actually Bamboo.  This is a common issue for any process that isn't running in an interactive console: https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe

            We added:

            stdbuf -i0 -o0 -e0
            

            To our long-running commands and it solved our issue.

            Nicholas Parrish added a comment - For future people that run into this:  we though we were having trouble with bamboo buffering the output of our jobs, but it wasn't actually Bamboo.  This is a common issue for any process that isn't running in an interactive console: https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe We added: stdbuf -i0 -o0 -e0 To our long-running commands and it solved our issue.

            +1

            Shane Wignall added a comment - +1

            Posted the above comment under Teo (using a shared company account).  email me if any info is needed.  

            Deleted Account (Inactive) added a comment - Posted the above comment under Teo (using a shared company account).  email me if any info is needed.  

            Totally agree with the above.  I have a job that runs completely when executed form the xml-data/build-dir/<JOB>/ and only takes 4min.  Output from it is about 400-500 lines of text to stdout.  When I run from Bamboo, it says there is no log file and does not show any output.  Unable to easily debug this, now I'm off to waste a ton of time trying to figure out what is happening.

            Deleted Account (Inactive) added a comment - Totally agree with the above.  I have a job that runs completely when executed form the xml-data/build-dir/<JOB>/ and only takes 4min.  Output from it is about 400-500 lines of text to stdout.  When I run from Bamboo, it says there is no log file and does not show any output.  Unable to easily debug this, now I'm off to waste a ton of time trying to figure out what is happening.

            I'd like to add that this isn't just annoying. It's downright confusing, and before I realized this was a bug, I lost a lot of time trying to troubleshoot a failed job. It's been over 7 years. Could someone please check into this?

            Deleted Account (Inactive) added a comment - I'd like to add that this isn't just annoying. It's downright confusing, and before I realized this was a bug, I lost a lot of time trying to troubleshoot a failed job. It's been over 7 years. Could someone please check into this?

            This is still a problem on the latest bamboo version. 

            Executables output is completely buffered. To the point where on a script that take 30 minutes, I will not see any output until the script is completed.

            (This is running on a local agent)

            It is REALLY annoying that this has not been addressed since 2011... 

            Fred Vassard added a comment - This is still a problem on the latest bamboo version.  Executables output is completely buffered. To the point where on a script that take 30 minutes, I will not see any output until the script is completed. (This is running on a local agent) It is REALLY annoying that this has not been addressed since 2011... 

              Unassigned Unassigned
              b08929f5741e Terris Linenbach
              Votes:
              16 Vote for this issue
              Watchers:
              15 Start watching this issue

                Created:
                Updated: