Issue Summary

When complex label regex is used, it will cause

• high CPU usage
• all other builds hung in Bamboo server (even though remote agents have completed the builds)
• builds that have completed on an agent earlier appear to still be running (because their result is being processed by the server, or waiting to be processed)

Steps to Reproduce

1. Create a build with complex label regex setting at Plan Configuration >> Stages & jobs >> <jobName> >> Miscellaneous/Other

   (.*,\([^abc]*\):.*)
   

2. Configure a Script task with the following command:

   yes "HelloWorldHelloWorldHelloWorldHelloWorldHelloWorldHelloWorldHelloWorldHelloWorldHelloWorldHelloWorldHelloWorldHelloWorld" | head -n 2000000
   

3. Create multiple remote agents
4. Disable local agent
5. Enable Concurrent builds
6. Run the same plan for a multiple times to perform concurrent builds

Expected Results

All builds will finish without problem

Actual Results

• High CPU usage in Bamboo server when the first plan finish running
• Some builds hung for certain time even though the remote agent logs shows that it has completed
• Some builds fail due to remote agent disconnected even though the remote agent logs shows that it has completed and sent the result to Bamboo server
• Thread dump shows that "BuildTailMessageProcessingThread-expensive:pool" is processing label search, causing it unable to process other remote agent communication messages:

  "9-BuildTailMessageProcessingThread-expensive:pool-16-thread-4196" #3245944 daemon prio=1 os_prio=0 tid=0x00007f28a014e800 nid=0x1e9f runnable [0x00007f277bc40000]
     java.lang.Thread.State: RUNNABLE
  	at java.util.regex.Pattern$CharProperty.match(Pattern.java:3776)
  	at java.util.regex.Pattern$Curly.match0(Pattern.java:4260)
  	at java.util.regex.Pattern$Curly.match(Pattern.java:4234)
  	at java.util.regex.Pattern$Branch.match(Pattern.java:4604)
  	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4658)
  	at java.util.regex.Pattern$Start.match(Pattern.java:3461)
  	at java.util.regex.Matcher.search(Matcher.java:1248)
  	at java.util.regex.Matcher.find(Matcher.java:637)
  	at com.atlassian.bamboo.plugins.labeller.BuildLabeller.resolveLabels(BuildLabeller.java:119)
  	at com.atlassian.bamboo.plugins.labeller.BuildLabeller.labelBuildResult(BuildLabeller.java:263)
  	at com.atlassian.bamboo.plugins.labeller.BuildLabeller.call(BuildLabeller.java:85)
  	at sun.reflect.GeneratedMethodAccessor1913.invoke(Unknown Source)
  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.lang.reflect.Method.invoke(Method.java:498)
  	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
  	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
  	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
  	at org.springframework.orm.hibernate5.support.OpenSessionInterceptor.invoke(OpenSessionInterceptor.java:93)
  	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
  	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
  	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
  	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
  

Notes

• If large build log (~160MB) is produced, the label search might take a few days to complete causing all other builds to hung and fail (Bamboo server have to restarted to stop this process)
• The fix in BAM-20570 may produce more than one thread processing the remote agent message but it will still produce high CPU and the same problem will still happen if multiple build produce large build logs at the same time
• It would be great if one of the following is used to prevent it:
  • timeout on label search so that it will not cause other builds to hung (user may not know that label search has timeout without checking Bamboo log)
  • label search will only process the first XX lines (the label setting can have a message that it will only process the first XX lines)
  • detect complex regex or initial ".*" (if it is possible) and prevent user from configuring it
  • perform label search in the remote agent side

Workaround

Do not use complex regex in label setting