Disabling a system scheduled job continues to run post restart of Confluence

Summary

Confluence uses Scheduled Jobs to automatize processes that are required to function properly. It's possible to enable/disable these jobs (managed jobs).

However, it was identified that Confluence will not respect these changes if we restart Confluence due to a code bug:

    @EventListener
    public void onApplicationStartedEvent(ApplicationStartedEvent event) {
        eventListenerRegistrar.unregister(this);

        // We need this variable; we can't simply use LifecycleManager#isStartedUp
        // because it still returns false when ApplicationStartedEvent is fired
        // (see DefaultLifecycleManager)
        receivedApplicationStartedEvent = true;

        scheduleUnmanagedJob();
    }

private void scheduleUnmanagedJob() {
        // This module won't receive ApplicationStartedEvent if it is installed after the application has been started
        boolean applicationStarted = receivedApplicationStartedEvent || lifecycleManager.isStartedUp();

        // Unmanaged job can only be scheduled after this module has been enabled and the application has been started
        if (!isEnabled() || !applicationStarted) {
            return;
        }

        Optional<JobRunner> jobRunner = getJobRunner();
        if (!jobRunner.isPresent()) {
            return;
        }

        schedulerService.registerJobRunner(getJobRunnerKey(), jobRunner.get());
        try {
            schedulerService.scheduleJob(getJobId(), newJobConfig());
        } catch (SchedulerServiceException e) {
            log.error("Could not schedule job, jobId: ", getJobId(), e);
        }
    }

This code intends to schedule for the unmanaged jobs but it doesn't check if the current job is managed or unmanaged before scheduling. Basically, disabling a system scheduled job will continue to run post restart of Confluence.

Steps to Reproduce

We will use the scheduled job cleanupTrigger as an example.

1. Install Confluence 7.1.2
2. Let the cleanupTrigger run once
3. Then disable it and check the database:

   select * FROM BANDANA where BANDANAKEY like '%cleanup%';
   
   bandanaid bandanacontext                                              bandanakey             bandanavalue                                                   
   --------- ----------------------------------------------------------- ---------------------- -------------------------------------------------------------- 
   86        com.atlassian.confluence.schedule.ScheduledJobConfiguration DEFAULT#cleanupTrigger <com.atlassian.confluence.schedule.ScheduledJobConfiguration>
     <enabled>
       <value>0</value>
     </enabled>
     <cronSchedule/>
     <repeatInterval/>
   </com.atlassian.confluence.schedule.ScheduledJobConfiguration>
   

4. Shutdown Confluence
5. Set the server time to ~1:50 am
6. add to log4j.properties the following properties to track the job execution upon application restart:

   log4j.logger.com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService=DEBUG
   log4j.logger.com.atlassian.confluence.impl.schedule.caesium.ConfluenceSchedulerService
   

7. Start Confluence - we should have the following in the logs indicating that the job is enqueued to run at the default scheduled time:

   2020-02-14 01:54:10,535 INFO [Catalina-utility-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 7.1.2 [build 8301 based on commit hash 13280cd41a592ca5883bbc1ae415b58671505365] - synchrony version 3.1.0-master-0f086db5
   ...
   ..
   2020-02-14 01:55:28,303 DEBUG [Catalina-utility-1] [scheduler.caesium.impl.CaesiumSchedulerService] unscheduleJob unscheduleJob for non-existent jobId: cleanupTrigger
   ...
   ..
   2020-02-14 01:55:29,993 DEBUG [Catalina-utility-1] [scheduler.caesium.impl.CaesiumSchedulerService] scheduleJob scheduleJob: cleanupTrigger: JobConfig[jobRunnerKey=com.atlassian.confluence.extra.officeconnector:cacheCleanupJob,runMode=RUN_LOCALLY,schedule=Schedule[type=CRON_EXPRESSION,cronScheduleInfo=CronScheduleInfo[cronExpression='0 0 2 * * ?',timeZone=Australia/Sydney]],parameters={com.atlassian.confluence.schedule.TenantAwareJobRescheduler:is-tz-sensitive=true}]
   2020-02-14 01:55:29,994 DEBUG [Catalina-utility-1] [scheduler.caesium.impl.CaesiumSchedulerService] enqueueJob Enqueued job 'cleanupTrigger' for Fri Feb 14 02:00:00 AEDT 2020
   ...
   ..
   (and then it runs at 02:00am)
   ...
   ..
   2020-02-14 02:00:00,143 DEBUG [Caesium-1-4] [scheduler.caesium.impl.CaesiumSchedulerService] enqueueJob Enqueued job 'cleanupTrigger' for Sat Feb 15 02:00:00 AEDT 2020
   

Expected Results

A managed Scheduled Job will remain disabled.

Actual Results

A disabled managed Scheduled Job will be rescheduled to run after a restart.

Workaround

This bug was identified and tested with the cleanupTrigger job (Office Connector). Follow the workaround from the related bug for this job:

• CONFSERVER-58957 - Preview macros for PDF and PowerPoint stops working after sometime

For other jobs, any new workaround found in the near-future will be provided in this report.