-
Bug
-
Resolution: Unresolved
-
Low
-
None
-
5.10.3
-
3
-
Severity 3 - Minor
-
2
-
Summary
It looks like some scheduled builds will randomly be droped with the message:
2017-03-22 01:02:02,668 INFO [10-BAM::PlanExec:pool-17-thread-3] [ExecutionLimitsServiceImpl] : PLAN-KEY cannot be run because the system is PAUSING
Steps to Reproduce
- Configure multiple scheduled builds to run during the time a scheduled backup is running
- Wait for the builds to happen after the backup is finished
Expected Results
Bamboo will run the scheduled builds after the backup is finished
Actual Results
Bamboo will sometimes try running buids when the server is still PAUSED and it will just avoid running those
Some log examples of this problem:
2017-03-21 01:06:39,027 INFO [scheduler_Worker-10] [XmlMigrator] Export completed. 01:06:39.021 2017-03-21 01:06:39,041 INFO [scheduler_Worker-6] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-FTTHC 2017-03-21 01:06:39,041 INFO [scheduler_Worker-8] [QuartzTriggerJob] Running quartz strategy job for CHAIN:TLR-FAT 2017-03-21 01:06:39,041 INFO [scheduler_Worker-10] [ServerLifecycleManagerImpl] Server state changed to 'RUNNING' from 'PAUSING' 2017-03-21 01:06:39,042 INFO [scheduler_Worker-3] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-UFTD 2017-03-21 01:06:39,042 INFO [scheduler_Worker-2] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-PIT 2017-03-21 01:06:39,044 INFO [10-BAM::PlanExec:pool-17-thread-3] [ScheduledBuildDetectionAction] Detecting changes for TLR-FAT... 2017-03-21 01:06:39,045 INFO [10-BAM::PlanExec:pool-17-thread-1] [ScheduledBuildDetectionAction] Detecting changes for EDC-UFTD... 2017-03-21 01:06:39,046 INFO [10-BAM::PlanExec:pool-17-thread-4] [ScheduledBuildDetectionAction] Detecting changes for EDC-FTTHC... 2017-03-21 01:06:39,046 INFO [10-BAM::PlanExec:pool-17-thread-2] [ScheduledBuildDetectionAction] Detecting changes for EDC-PIT... 2017-03-21 01:06:39,051 INFO [10-BAM::PlanExec:pool-17-thread-2] [ScheduledBuildDetectionAction] : Schedule build EDC-PIT-1690 triggered 2017-03-21 01:06:39,052 INFO [10-BAM::PlanExec:pool-17-thread-1] [ScheduledBuildDetectionAction] : Schedule build EDC-UFTD-475 triggered 2017-03-21 01:06:39,053 INFO [10-BAM::PlanExec:pool-17-thread-3] [ScheduledBuildDetectionAction] : Schedule build TLR-FAT-402 triggered 2017-03-21 01:06:39,066 INFO [10-BAM::PlanExec:pool-17-thread-4] [ScheduledBuildDetectionAction] : Schedule build EDC-FTTHC-413 triggered -- -- 2017-03-22 01:02:02,650 INFO [scheduler_Worker-7] [XmlMigrator] Export completed. 01:02:02.644 2017-03-22 01:02:02,668 INFO [scheduler_Worker-1] [QuartzTriggerJob] Running quartz strategy job for CHAIN:TLR-FAT 2017-03-22 01:02:02,668 INFO [scheduler_Worker-9] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-PIT 2017-03-22 01:02:02,668 INFO [scheduler_Worker-5] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-UFTD 2017-03-22 01:02:02,668 INFO [scheduler_Worker-3] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-FTTHC 2017-03-22 01:02:02,668 INFO [10-BAM::PlanExec:pool-17-thread-3] [ExecutionLimitsServiceImpl] : TLR-FAT cannot be run because the system is PAUSING 2017-03-22 01:02:02,669 INFO [10-BAM::PlanExec:pool-17-thread-3] [ExecutionLimitsServiceImpl] : EDC-PIT cannot be run because the system is PAUSING 2017-03-22 01:02:02,669 INFO [10-BAM::PlanExec:pool-17-thread-3] [ExecutionLimitsServiceImpl] : EDC-UFTD cannot be run because the system is PAUSING 2017-03-22 01:02:02,669 INFO [10-BAM::PlanExec:pool-17-thread-3] [ExecutionLimitsServiceImpl] : EDC-FTTHC cannot be run because the system is PAUSING 2017-03-22 01:02:02,669 INFO [10-BAM::PlanExec:pool-17-thread-4] [ExecutionLimitsServiceImpl] : ZZZ-BD48 cannot be run because the system is PAUSING 2017-03-22 01:02:02,670 INFO [10-BAM::PlanExec:pool-17-thread-2] [ExecutionLimitsServiceImpl] : TLR-TCA1 cannot be run because the system is PAUSING 2017-03-22 01:02:02,671 INFO [scheduler_Worker-7] [ServerLifecycleManagerImpl] Server state changed to 'RUNNING' from 'PAUSING' -- -- 2017-03-23 01:02:10,415 INFO [scheduler_Worker-6] [XmlMigrator] Export completed. 01:02:10.411 2017-03-23 01:02:10,427 INFO [scheduler_Worker-5] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-FTTHC 2017-03-23 01:02:10,428 INFO [scheduler_Worker-4] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-UFTD 2017-03-23 01:02:10,428 INFO [scheduler_Worker-8] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-PIT 2017-03-23 01:02:10,428 INFO [scheduler_Worker-10] [QuartzTriggerJob] Running quartz strategy job for CHAIN:EDC-UPT 2017-03-23 01:02:10,428 INFO [10-BAM::PlanExec:pool-17-thread-4] [ExecutionLimitsServiceImpl] : ZZZ-BD48 cannot be run because the system is PAUSING 2017-03-23 01:02:10,428 INFO [scheduler_Worker-9] [QuartzTriggerJob] Running quartz strategy job for CHAIN:TLR-FAT 2017-03-23 01:02:10,430 INFO [scheduler_Worker-6] [ServerLifecycleManagerImpl] Server state changed to 'RUNNING' from 'PAUSING' 2017-03-23 01:02:10,432 INFO [10-BAM::PlanExec:pool-17-thread-4] [ScheduledBuildDetectionAction] Detecting changes for EDC-FTTHC... 2017-03-23 01:02:10,434 INFO [10-BAM::PlanExec:pool-17-thread-1] [ScheduledBuildDetectionAction] Detecting changes for EDC-PIT... 2017-03-23 01:02:10,434 INFO [10-BAM::PlanExec:pool-17-thread-3] [ScheduledBuildDetectionAction] Detecting changes for EDC-UPT... 2017-03-23 01:02:10,437 INFO [10-BAM::PlanExec:pool-17-thread-2] [ScheduledBuildDetectionAction] Detecting changes for EDC-UFTD... 2017-03-23 01:02:10,438 INFO [10-BAM::PlanExec:pool-17-thread-4] [ScheduledBuildDetectionAction] : Schedule build EDC-FTTHC-434 triggered 2017-03-23 01:02:10,439 INFO [10-BAM::PlanExec:pool-17-thread-3] [ScheduledBuildDetectionAction] : Schedule build EDC-UPT-2988 triggered 2017-03-23 01:02:10,442 INFO [10-BAM::PlanExec:pool-17-thread-1] [ScheduledBuildDetectionAction] : Schedule build EDC-PIT-1693 triggered 2017-03-23 01:02:10,445 INFO [10-BAM::PlanExec:pool-17-thread-2] [ScheduledBuildDetectionAction] : Schedule build EDC-UFTD-476 triggered
In this case, the plan TLR-FAT was configured to run at 00:05 AM. Backup starts at 00:00 AM so usually the plan is built after 1 AM when the export is finished, but sometimes it is dropped and not run.
Workaround
Option 1 - Configure an external backup tool and avoid using the scheduled backup
Option 2 - Do not configure builds to run when the server is running the scheduled backup