[BAM-22017] Bamboo throws FileNotFoundException when attempting to restore state of the build queue

Type: Bug
Resolution: Fixed
Priority: Low
Fix Version/s: 9.0.0
Affects Version/s: 8.2.5
Component/s: Build Queues
Labels:
None

Symptom Severity:
Severity 3 - Minor
Bug Fix Policy:
View Atlassian Server bug fix policy

colored texth3. Issue Summary

Bamboo throws FileNotFoundException when trying to read from files inside the <bamboo-shared-home>/serverState/queue directory in an attempt restore the state of the queue. This is part of the build resiliency feature in Bamboo Data Center.

The issue can also happen during regular operation if queue is expected to be offloaded to the disk, but a file is missing.

This affects Data Center:

Steps to Reproduce

N/A

Expected Results

N/A

Actual Results

Bamboo is unable to access the file and throws the following exception in the <bamboo-home>/logs/atlassian-bamboo.log file:

2022-11-07 10:10:25,109 ERROR [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-52] [AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [AgentConfigurationUpdatedEvent Elastic Agent on i-09c12e59d32470f34] from the invoker [com.atlassian.event.internal.ComparableListenerInvoker@faffd7d]
java.lang.RuntimeException: Unexpected exception. Listener: com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener event: com.atlassian.bamboo.event.AgentConfigurationUpdatedEvent
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:53)
        at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Unexpected exception
        at com.atlassian.bamboo.util.BambooObjectUtils.asRuntimeException(BambooObjectUtils.java:119)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:70)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.load(CommonContextMaps.java:112)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.get(CommonContextMaps.java:91)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.lambda$recalculateExecutables$1(AbstractQueueOfExecutables.java:191)
        at com.atlassian.bamboo.utils.scopedcaches.ThreadScopedCaches.inCachingScope(ThreadScopedCaches.java:56)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.recalculateExecutables(AbstractQueueOfExecutables.java:184)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.lambda$invalidateExecutables$3(BuildQueueManagerImpl.java:285)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.invalidateExecutables(BuildQueueManagerImpl.java:284)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener.onAgentConfigurationUpdatedEvent(BuildQueueUpdateListener.java:21)
        at sun.reflect.GeneratedMethodAccessor1749.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
        ... 9 more
Caused by: java.io.FileNotFoundException: /bamboo-shared-home/serverState/queue/PROJ-PLAN-JOB1-13 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:63)
        ... 22 more

Workaround

Disable the build resiliency feature to prevent Bamboo from attempting to restore the state of the queue upon a restart.

Stop Bamboo.
Add the -Dbamboo.ignore.server.state.on.restart=true to the Bamboo JVM by following the "Configuring your system properties" documentation.
Start Bamboo.
Bamboo should no longer attempt to restore the queue and you should no longer see the exception above in the application logs.

Notes

Elastic agents

The exception above might result in Bamboo attempting to spin up a new elastic agent every time it tries to restore the job whose file is missing from the <bamboo-shared-home>/serverState/queue folder.

is duplicated by

BAM-22069 When agents are restarted:Build jobs(which are queued while the agent is offline) not picked up by agent even if the agent is in the Idle state and eligible for executing the build.

Closed

is related to: BDEV-17329 Failed to load

mentioned in: Page Failed to load

Jyothi Charupalli made changes - 13/Jan/2023 12:47 PM

Description

Original: h3. Issue Summary

Bamboo throws *FileNotFoundException* when trying to read from files inside the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue directory in an attempt restore the state of the queue. This is part of the [build resiliency|https://confluence.atlassian.com/bamboo/build-resiliency-in-bamboo-data-center-1063170557.html] feature in Bamboo Data Center.

The issue can also happen during regular operation if queue is expected to be offloaded to the disk, but a file is missing.

This affects Data Center: (/)

h3. Steps to Reproduce

N/A

h3. Expected Results

N/A

h3. Actual Results

Bamboo is unable to access the file and throws the following exception in the <[bamboo-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/logs/atlassian-bamboo.log file:

{noformat}
2022-11-07 10:10:25,109 ERROR [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-52] [AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [AgentConfigurationUpdatedEvent Elastic Agent on i-09c12e59d32470f34] from the invoker [com.atlassian.event.internal.ComparableListenerInvoker@faffd7d]
java.lang.RuntimeException: Unexpected exception. Listener: com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener event: com.atlassian.bamboo.event.AgentConfigurationUpdatedEvent
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:53)
        at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Unexpected exception
        at com.atlassian.bamboo.util.BambooObjectUtils.asRuntimeException(BambooObjectUtils.java:119)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:70)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.load(CommonContextMaps.java:112)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.get(CommonContextMaps.java:91)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.lambda$recalculateExecutables$1(AbstractQueueOfExecutables.java:191)
        at com.atlassian.bamboo.utils.scopedcaches.ThreadScopedCaches.inCachingScope(ThreadScopedCaches.java:56)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.recalculateExecutables(AbstractQueueOfExecutables.java:184)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.lambda$invalidateExecutables$3(BuildQueueManagerImpl.java:285)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.invalidateExecutables(BuildQueueManagerImpl.java:284)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener.onAgentConfigurationUpdatedEvent(BuildQueueUpdateListener.java:21)
        at sun.reflect.GeneratedMethodAccessor1749.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
        ... 9 more
Caused by: java.io.FileNotFoundException: /bamboo-shared-home/serverState/queue/PROJ-PLAN-JOB1-13 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:63)
        ... 22 more
{noformat}

h3. Workaround

Disable the build resiliency feature to prevent Bamboo from attempting to restore the state of the queue upon a restart.
# Stop Bamboo.
# Add the {{-Dbamboo.ignore.server.state.on.restart=true}} to the Bamboo JVM by following the ["Configuring your system properties"|https://confluence.atlassian.com/bamboo/configuring-your-system-properties-289277345.html] documentation.
# Start Bamboo.
# Bamboo should no longer attempt to restore the queue and you should no longer see the exception above in the application logs.

h3. Notes

{panel:title=(!) Elastic agents|borderStyle=dashed|borderWidth=2px|borderColor=#6a8eb3|titleBGColor=#f8f8f8|bgColor=#f8f8f8}
The exception above might result in Bamboo attempting to spin up a new elastic agent every time it tries to restore the job whose file is missing from the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue folder.
{panel}

New: {color:red}colored text{color}h3. Issue Summary

Bamboo throws *FileNotFoundException* when trying to read from files inside the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue directory in an attempt restore the state of the queue. This is part of the [build resiliency|https://confluence.atlassian.com/bamboo/build-resiliency-in-bamboo-data-center-1063170557.html] feature in Bamboo Data Center.

The issue can also happen during regular operation if queue is expected to be offloaded to the disk, but a file is missing.

This affects Data Center: (/)

h3. Steps to Reproduce

N/A

h3. Expected Results

N/A

h3. Actual Results

Bamboo is unable to access the file and throws the following exception in the <[bamboo-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/logs/atlassian-bamboo.log file:

{noformat}
2022-11-07 10:10:25,109 ERROR [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-52] [AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [AgentConfigurationUpdatedEvent Elastic Agent on i-09c12e59d32470f34] from the invoker [com.atlassian.event.internal.ComparableListenerInvoker@faffd7d]
java.lang.RuntimeException: Unexpected exception. Listener: com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener event: com.atlassian.bamboo.event.AgentConfigurationUpdatedEvent
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:53)
        at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Unexpected exception
        at com.atlassian.bamboo.util.BambooObjectUtils.asRuntimeException(BambooObjectUtils.java:119)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:70)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.load(CommonContextMaps.java:112)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.get(CommonContextMaps.java:91)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.lambda$recalculateExecutables$1(AbstractQueueOfExecutables.java:191)
        at com.atlassian.bamboo.utils.scopedcaches.ThreadScopedCaches.inCachingScope(ThreadScopedCaches.java:56)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.recalculateExecutables(AbstractQueueOfExecutables.java:184)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.lambda$invalidateExecutables$3(BuildQueueManagerImpl.java:285)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.invalidateExecutables(BuildQueueManagerImpl.java:284)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener.onAgentConfigurationUpdatedEvent(BuildQueueUpdateListener.java:21)
        at sun.reflect.GeneratedMethodAccessor1749.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
        ... 9 more
Caused by: java.io.FileNotFoundException: /bamboo-shared-home/serverState/queue/PROJ-PLAN-JOB1-13 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:63)
        ... 22 more
{noformat}

h3. Workaround

Disable the build resiliency feature to prevent Bamboo from attempting to restore the state of the queue upon a restart.
# Stop Bamboo.
# Add the {{-Dbamboo.ignore.server.state.on.restart=true}} to the Bamboo JVM by following the ["Configuring your system properties"|https://confluence.atlassian.com/bamboo/configuring-your-system-properties-289277345.html] documentation.
# Start Bamboo.
# Bamboo should no longer attempt to restore the queue and you should no longer see the exception above in the application logs.

h3. Notes

{panel:title=(!) Elastic agents|borderStyle=dashed|borderWidth=2px|borderColor=#6a8eb3|titleBGColor=#f8f8f8|bgColor=#f8f8f8}
The exception above might result in Bamboo attempting to spin up a new elastic agent every time it tries to restore the job whose file is missing from the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue folder.
{panel}

Marcin Gardias made changes - 13/Jan/2023 12:29 PM

Description

Original: h3. Issue Summary

Bamboo throws *FileNotFoundException* when trying to read from files inside the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue directory in an attempt restore the state of the queue. This is part of the [build resiliency|https://confluence.atlassian.com/bamboo/build-resiliency-in-bamboo-data-center-1063170557.html] feature in Bamboo Data Center.

This affects Data Center: (/)

h3. Steps to Reproduce

N/A

h3. Expected Results

N/A

h3. Actual Results

Bamboo is unable to access the file and throws the following exception in the <[bamboo-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/logs/atlassian-bamboo.log file:

{noformat}
2022-11-07 10:10:25,109 ERROR [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-52] [AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [AgentConfigurationUpdatedEvent Elastic Agent on i-09c12e59d32470f34] from the invoker [com.atlassian.event.internal.ComparableListenerInvoker@faffd7d]
java.lang.RuntimeException: Unexpected exception. Listener: com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener event: com.atlassian.bamboo.event.AgentConfigurationUpdatedEvent
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:53)
        at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Unexpected exception
        at com.atlassian.bamboo.util.BambooObjectUtils.asRuntimeException(BambooObjectUtils.java:119)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:70)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.load(CommonContextMaps.java:112)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.get(CommonContextMaps.java:91)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.lambda$recalculateExecutables$1(AbstractQueueOfExecutables.java:191)
        at com.atlassian.bamboo.utils.scopedcaches.ThreadScopedCaches.inCachingScope(ThreadScopedCaches.java:56)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.recalculateExecutables(AbstractQueueOfExecutables.java:184)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.lambda$invalidateExecutables$3(BuildQueueManagerImpl.java:285)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.invalidateExecutables(BuildQueueManagerImpl.java:284)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener.onAgentConfigurationUpdatedEvent(BuildQueueUpdateListener.java:21)
        at sun.reflect.GeneratedMethodAccessor1749.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
        ... 9 more
Caused by: java.io.FileNotFoundException: /bamboo-shared-home/serverState/queue/PROJ-PLAN-JOB1-13 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:63)
        ... 22 more
{noformat}

h3. Workaround

Disable the build resiliency feature to prevent Bamboo from attempting to restore the state of the queue upon a restart.
# Stop Bamboo.
# Add the {{-Dbamboo.ignore.server.state.on.restart=true}} to the Bamboo JVM by following the ["Configuring your system properties"|https://confluence.atlassian.com/bamboo/configuring-your-system-properties-289277345.html] documentation.
# Start Bamboo.
# Bamboo should no longer attempt to restore the queue and you should no longer see the exception above in the application logs.

h3. Notes

{panel:title=(!) Elastic agents|borderStyle=dashed|borderWidth=2px|borderColor=#6a8eb3|titleBGColor=#f8f8f8|bgColor=#f8f8f8}
The exception above might result in Bamboo attempting to spin up a new elastic agent every time it tries to restore the job whose file is missing from the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue folder.
{panel}

New: h3. Issue Summary

Bamboo throws *FileNotFoundException* when trying to read from files inside the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue directory in an attempt restore the state of the queue. This is part of the [build resiliency|https://confluence.atlassian.com/bamboo/build-resiliency-in-bamboo-data-center-1063170557.html] feature in Bamboo Data Center.

The issue can also happen during regular operation if queue is expected to be offloaded to the disk, but a file is missing.

This affects Data Center: (/)

h3. Steps to Reproduce

N/A

h3. Expected Results

N/A

h3. Actual Results

Bamboo is unable to access the file and throws the following exception in the <[bamboo-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/logs/atlassian-bamboo.log file:

{noformat}
2022-11-07 10:10:25,109 ERROR [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-52] [AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [AgentConfigurationUpdatedEvent Elastic Agent on i-09c12e59d32470f34] from the invoker [com.atlassian.event.internal.ComparableListenerInvoker@faffd7d]
java.lang.RuntimeException: Unexpected exception. Listener: com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener event: com.atlassian.bamboo.event.AgentConfigurationUpdatedEvent
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:53)
        at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Unexpected exception
        at com.atlassian.bamboo.util.BambooObjectUtils.asRuntimeException(BambooObjectUtils.java:119)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:70)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.load(CommonContextMaps.java:112)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.get(CommonContextMaps.java:91)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.lambda$recalculateExecutables$1(AbstractQueueOfExecutables.java:191)
        at com.atlassian.bamboo.utils.scopedcaches.ThreadScopedCaches.inCachingScope(ThreadScopedCaches.java:56)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.recalculateExecutables(AbstractQueueOfExecutables.java:184)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.lambda$invalidateExecutables$3(BuildQueueManagerImpl.java:285)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.invalidateExecutables(BuildQueueManagerImpl.java:284)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener.onAgentConfigurationUpdatedEvent(BuildQueueUpdateListener.java:21)
        at sun.reflect.GeneratedMethodAccessor1749.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
        ... 9 more
Caused by: java.io.FileNotFoundException: /bamboo-shared-home/serverState/queue/PROJ-PLAN-JOB1-13 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:63)
        ... 22 more
{noformat}

h3. Workaround

Disable the build resiliency feature to prevent Bamboo from attempting to restore the state of the queue upon a restart.
# Stop Bamboo.
# Add the {{-Dbamboo.ignore.server.state.on.restart=true}} to the Bamboo JVM by following the ["Configuring your system properties"|https://confluence.atlassian.com/bamboo/configuring-your-system-properties-289277345.html] documentation.
# Start Bamboo.
# Bamboo should no longer attempt to restore the queue and you should no longer see the exception above in the application logs.

h3. Notes

{panel:title=(!) Elastic agents|borderStyle=dashed|borderWidth=2px|borderColor=#6a8eb3|titleBGColor=#f8f8f8|bgColor=#f8f8f8}
The exception above might result in Bamboo attempting to spin up a new elastic agent every time it tries to restore the job whose file is missing from the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue folder.
{panel}

Marcin Gardias made changes - 13/Jan/2023 12:27 PM

Link

New: This issue is duplicated by ~~BAM-22069~~ [ ~~BAM-22069~~ ]

Jyothi Charupalli made changes - 04/Jan/2023 12:10 PM

Remote Link

Original: This issue links to "Page (Confluence)" [ 718737 ]

Jyothi Charupalli made changes - 04/Jan/2023 12:06 PM

Remote Link

New: This issue links to "Page (Confluence)" [ 718737 ]

Shashank Kumar made changes - 16/Nov/2022 7:21 AM

Remote Link

New: This issue links to "Page (Confluence)" [ 706798 ]

Bruno Rosa made changes - 11/Nov/2022 7:51 AM

Summary

Original: Bamboo throws FileNotFoundException when attempting to restore state of the queue

New: Bamboo throws FileNotFoundException when attempting to restore state of the build queue

Bruno Rosa made changes - 10/Nov/2022 10:16 AM

Description

Original: h3. Issue Summary

Bamboo throws *FileNotFoundException* when trying to read from files inside the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue directory in an attempt restore the state of the queue. This is part of the [build resiliency|https://confluence.atlassian.com/bamboo/build-resiliency-in-bamboo-data-center-1063170557.html] feature in Bamboo Data Center.

This is reproducible on Data Center: (/)

h3. Steps to Reproduce

N/A

h3. Expected Results

N/A

h3. Actual Results

Bamboo is unable to access the file and throws the following exception in the <[bamboo-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/logs/atlassian-bamboo.log file:

{noformat}
2022-11-07 10:10:25,109 ERROR [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-52] [AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [AgentConfigurationUpdatedEvent Elastic Agent on i-09c12e59d32470f34] from the invoker [com.atlassian.event.internal.ComparableListenerInvoker@faffd7d]
java.lang.RuntimeException: Unexpected exception. Listener: com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener event: com.atlassian.bamboo.event.AgentConfigurationUpdatedEvent
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:53)
        at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Unexpected exception
        at com.atlassian.bamboo.util.BambooObjectUtils.asRuntimeException(BambooObjectUtils.java:119)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:70)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.load(CommonContextMaps.java:112)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.get(CommonContextMaps.java:91)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.lambda$recalculateExecutables$1(AbstractQueueOfExecutables.java:191)
        at com.atlassian.bamboo.utils.scopedcaches.ThreadScopedCaches.inCachingScope(ThreadScopedCaches.java:56)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.recalculateExecutables(AbstractQueueOfExecutables.java:184)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.lambda$invalidateExecutables$3(BuildQueueManagerImpl.java:285)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.invalidateExecutables(BuildQueueManagerImpl.java:284)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener.onAgentConfigurationUpdatedEvent(BuildQueueUpdateListener.java:21)
        at sun.reflect.GeneratedMethodAccessor1749.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
        ... 9 more
Caused by: java.io.FileNotFoundException: /bamboo-shared-home/serverState/queue/PROJ-PLAN-JOB1-13 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:63)
        ... 22 more
{noformat}

h3. Workaround

Disable the build resiliency feature to prevent Bamboo from attempting to restore the state of the queue upon a restart.
# Stop Bamboo.
# Add the {{-Dbamboo.ignore.server.state.on.restart=true}} to the Bamboo JVM by following the ["Configuring your system properties"|https://confluence.atlassian.com/bamboo/configuring-your-system-properties-289277345.html] documentation.
# Start Bamboo.
# Bamboo should no longer attempt to restore the queue and you should no longer see the exception above in the application logs.

h3. Notes

{panel:title=(!) Elastic agents|borderStyle=dashed|borderWidth=2px|borderColor=#6a8eb3|titleBGColor=#f8f8f8|bgColor=#f8f8f8}
The exception above might result in Bamboo attempting to spin up a new elastic agent every time it tries to restore the job whose file is missing from the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue folder.
{panel}

New: h3. Issue Summary

Bamboo throws *FileNotFoundException* when trying to read from files inside the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue directory in an attempt restore the state of the queue. This is part of the [build resiliency|https://confluence.atlassian.com/bamboo/build-resiliency-in-bamboo-data-center-1063170557.html] feature in Bamboo Data Center.

This affects Data Center: (/)

h3. Steps to Reproduce

N/A

h3. Expected Results

N/A

h3. Actual Results

Bamboo is unable to access the file and throws the following exception in the <[bamboo-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/logs/atlassian-bamboo.log file:

{noformat}
2022-11-07 10:10:25,109 ERROR [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-52] [AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [AgentConfigurationUpdatedEvent Elastic Agent on i-09c12e59d32470f34] from the invoker [com.atlassian.event.internal.ComparableListenerInvoker@faffd7d]
java.lang.RuntimeException: Unexpected exception. Listener: com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener event: com.atlassian.bamboo.event.AgentConfigurationUpdatedEvent
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:53)
        at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
        at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$null$0(AsynchronousAbleEventDispatcher.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
        at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
        at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Unexpected exception
        at com.atlassian.bamboo.util.BambooObjectUtils.asRuntimeException(BambooObjectUtils.java:119)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:70)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.load(CommonContextMaps.java:112)
        at com.atlassian.bamboo.v2.build.queue.queues.CommonContextMaps$ForwardingOffloadingCommonContextMap.get(CommonContextMaps.java:91)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.lambda$recalculateExecutables$1(AbstractQueueOfExecutables.java:191)
        at com.atlassian.bamboo.utils.scopedcaches.ThreadScopedCaches.inCachingScope(ThreadScopedCaches.java:56)
        at com.atlassian.bamboo.v2.build.queue.queues.AbstractQueueOfExecutables.recalculateExecutables(AbstractQueueOfExecutables.java:184)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.lambda$invalidateExecutables$3(BuildQueueManagerImpl.java:285)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueManagerImpl.invalidateExecutables(BuildQueueManagerImpl.java:284)
        at com.atlassian.bamboo.v2.build.queue.BuildQueueUpdateListener.onAgentConfigurationUpdatedEvent(BuildQueueUpdateListener.java:21)
        at sun.reflect.GeneratedMethodAccessor1749.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
        ... 9 more
Caused by: java.io.FileNotFoundException: /bamboo-shared-home/serverState/queue/PROJ-PLAN-JOB1-13 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at com.atlassian.bamboo.v2.build.queue.queues.XStreamQueuePersisterImpl.load(XStreamQueuePersisterImpl.java:63)
        ... 22 more
{noformat}

h3. Workaround

Disable the build resiliency feature to prevent Bamboo from attempting to restore the state of the queue upon a restart.
# Stop Bamboo.
# Add the {{-Dbamboo.ignore.server.state.on.restart=true}} to the Bamboo JVM by following the ["Configuring your system properties"|https://confluence.atlassian.com/bamboo/configuring-your-system-properties-289277345.html] documentation.
# Start Bamboo.
# Bamboo should no longer attempt to restore the queue and you should no longer see the exception above in the application logs.

h3. Notes

{panel:title=(!) Elastic agents|borderStyle=dashed|borderWidth=2px|borderColor=#6a8eb3|titleBGColor=#f8f8f8|bgColor=#f8f8f8}
The exception above might result in Bamboo attempting to spin up a new elastic agent every time it tries to restore the job whose file is missing from the <[bamboo-shared-home|https://confluence.atlassian.com/bamboo/locating-important-directories-and-files-289277247.html]>/serverState/queue folder.
{panel}

Bruno Rosa made changes - 10/Nov/2022 10:08 AM

Resolution		New: Fixed [ 1 ]
Status	Original: Needs Triage [ 10030 ]	New: Closed [ 6 ]

Bruno Rosa made changes - 10/Nov/2022 10:07 AM

Fix Version/s

New: 9.0.0 [ 100790 ]

Assignee:: Marcin Gardias

Reporter:: Bruno Rosa

Affected customers:: 0 This affects my team

Watchers:: 1 Start watching this issue

Created:: 10/Nov/2022 10:05 AM

Updated:: 13/Jan/2023 12:47 PM

Resolved:: 10/Nov/2022 10:08 AM

Details

Description

Steps to Reproduce

Expected Results

Actual Results

Workaround

Notes

Attachments

Issue Links

Forms

Activity

People

Dates