Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-98287

Confluence startup blocks on "Securing local config secrets now"

      Issue Summary

      Confluence startup can block indefinitely on a virtual system in the absence of entropy random data.

      This is reproducible on Data Center: yes

      Steps to Reproduce

      1. Install Confluence 8.x on a VM with no entropy random data and fully start it up and shut it back down
      2. Upgrade to Confluence 9.1.0 but do not start Confluence yet
      3. Update <Confluence9.1.0Install>/confluence/WEB-INF/classes/log4j.properties with
        log4j.logger.com.atlassian.confluence.upgrade.upgradetask=INFO
        
      4. Start Confluence 9.1.0

      Expected Results

      The following lines are logged in atlassian-confluence.log file with no delay :

      2024-10-22 11:37:03,885 INFO [Catalina-utility-1] [atlassian.confluence.upgrade.UpgradeTask] secureLocalConfigSecrets Securing local config secrets now...
      2024-10-22 11:37:04,251 INFO [Catalina-utility-1] [atlassian.confluence.upgrade.UpgradeTask] secureLocalConfigSecrets Local config secrets secured.
      

      Actual Results

      Only the following line is logged in atlassian-confluence.log file:

      2024-10-22 11:37:03,885 INFO [Catalina-utility-1] [atlassian.confluence.upgrade.UpgradeTask] secureLocalConfigSecrets Securing local config secrets now...
      

      and Confluence startup blocks .

      Taking Thread dumps shows:

      "Catalina-utility-1" #20 prio=1 os_prio=0 cpu=37724.56ms elapsed=1677.10s tid=0x00007fa7ac897820 nid=0x8b runnable  [0x00007fa7360fc000]
         java.lang.Thread.State: RUNNABLE
      	at java.io.FileInputStream.readBytes(java.base@17.0.12/Native Method)
      	at java.io.FileInputStream.read(java.base@17.0.12/FileInputStream.java:276)
      	at java.io.FilterInputStream.read(java.base@17.0.12/FilterInputStream.java:132)
      	at sun.security.provider.NativePRNG$RandomIO.readFully(java.base@17.0.12/NativePRNG.java:425)
      	at sun.security.provider.NativePRNG$RandomIO.ensureBufferValid(java.base@17.0.12/NativePRNG.java:528)
      	at sun.security.provider.NativePRNG$RandomIO.implNextBytes(java.base@17.0.12/NativePRNG.java:547)
      	- locked <0x00000000c0378710> (a java.lang.Object)
      	at sun.security.provider.NativePRNG$Blocking.engineNextBytes(java.base@17.0.12/NativePRNG.java:269)
      	at java.security.SecureRandom.nextBytes(java.base@17.0.12/SecureRandom.java:758)
      	at com.atlassian.secrets.service.aes.AESEncryptionBackend.generateIV(AESEncryptionBackend.java:156)
      	at com.atlassian.secrets.service.aes.AESEncryptionBackend$$Lambda$2568/0x00007fa73dd659c8.get(Unknown Source)
      	at com.atlassian.secrets.service.aes.AESEncryptionBackend.seal(AESEncryptionBackend.java:102)
      	at com.atlassian.secrets.service.DefaultSecretService.put(DefaultSecretService.java:56)
      	at com.atlassian.confluence.impl.security.ConfluenceSecretService.put(ConfluenceSecretService.java:115)
      	at com.atlassian.confluence.impl.setup.ConfluenceApplicationConfig.lambda$save$1(ConfluenceApplicationConfig.java:135)
      	at com.atlassian.confluence.impl.setup.ConfluenceApplicationConfig$$Lambda$2539/0x00007fa73dd38658.accept(Unknown Source)
      	at java.lang.Iterable.forEach(java.base@17.0.12/Iterable.java:75)
      	at com.atlassian.confluence.impl.setup.ConfluenceApplicationConfig.save(ConfluenceApplicationConfig.java:123)
      	- locked <0x00000000c275a8f0> (a com.atlassian.confluence.impl.setup.ConfluenceApplicationConfig)
      	at com.atlassian.confluence.upgrade.upgradetask.SecureLocalConfigSecretsUpgradeTask.secureLocalConfigSecrets(SecureLocalConfigSecretsUpgradeTask.java:75)
      	at com.atlassian.confluence.upgrade.upgradetask.SecureLocalConfigSecretsUpgradeTask.upgrade(SecureLocalConfigSecretsUpgradeTask.java:65)
      	at com.atlassian.confluence.upgrade.upgradetask.SecureLocalConfigSecretsUpgradeTask.doUpgrade(SecureLocalConfigSecretsUpgradeTask.java:60)
      	at com.atlassian.confluence.upgrade.AbstractUpgradeManager$UpgradeStep$4.execute(AbstractUpgradeManager.java:788)
      	at com.atlassian.confluence.upgrade.AbstractUpgradeManager.executeUpgradeTask(AbstractUpgradeManager.java:325)
      	at com.atlassian.confluence.upgrade.AbstractUpgradeManager.executeUpgradeStep(AbstractUpgradeManager.java:296)
      	at com.atlassian.confluence.upgrade.AbstractUpgradeManager.runUpgradeTasks(AbstractUpgradeManager.java:267)
      	at com.atlassian.confluence.upgrade.impl.DefaultUpgradeManager.runUpgradeTasks(DefaultUpgradeManager.java:346)
      	at com.atlassian.confluence.upgrade.AbstractUpgradeManager.runAllUpgradeTasks(AbstractUpgradeManager.java:181)
      	at com.atlassian.confluence.upgrade.AbstractUpgradeManager.upgrade(AbstractUpgradeManager.java:140)
      ...
      

      Setting this JVM flag has no effect

      -Djava.security.egd=file:/dev/urandom
      

      Diagnosis

      Running this on the Linux OS will block indefinitely when the OS has insufficient entropy data :

      head -1 /dev/random
      

      Workaround

      If starting Confluence in a Docker container environment, re-map /dev/random to /dev/urandom, e.g.

      Docker run command
      -v /dev/urandom:/dev/random:ro
      

          Form Name

            [CONFSERVER-98287] Confluence startup blocks on "Securing local config secrets now"

            Basar Beykoz made changes -
            Link New: This issue is related to CONFSERVER-98637 [ CONFSERVER-98637 ]
            Lou Paglaiccetti made changes -
            Remote Link Original: This issue links to "Page (Confluence)" [ 965551 ]
            Lou Paglaiccetti made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 965551 ]
            SET Analytics Bot made changes -
            UIS Original: 13 New: 40
            Lou Paglaiccetti made changes -
            Assignee Original: Lou Paglaiccetti [ f040be51555f ] New: Adam Brokes [ abrokes ]
            Lou Paglaiccetti made changes -
            Assignee Original: Adam Brokes [ abrokes ] New: Lou Paglaiccetti [ f040be51555f ]
            Adam Brokes made changes -
            Assignee New: Adam Brokes [ abrokes ]
            Adam Brokes made changes -
            QA Demo Status Original: Not Done [ 14330 ] New: Not Needed [ 14332 ]
            QA Kickoff Status Original: Not Done [ 14234 ] New: Not Needed [ 14236 ]
            Resolution New: Fixed [ 1 ]
            Status Original: In Progress [ 3 ] New: Closed [ 6 ]
            Adam Brokes made changes -
            Fix Version/s New: 9.1.1 [ 109701 ]
            Engineering Taxonomy Bot made changes -
            Labels Original: internal-kickoff New: ewt-rtb-service-operations-and-tech-entropy internal-kickoff

              abrokes Adam Brokes
              hlam@atlassian.com Eric Lam
              Affected customers:
              0 This affects my team
              Watchers:
              13 Start watching this issue

                Created:
                Updated:
                Resolved: