Hi 873135d8df40 , thanks for your suggestions. In terms of this issue, we have already made some UI changes from LTS versions 7.13.17, 7.19.9,  8.3.0 that the User Macro Administration interface will list modules that are used but have not been added to the allow list, so to help administrators review what modules are added to ensure usage is in line with their expectations.

Given the number of comments here and since this issue arose in May of 2023 do we think it might be worthwhile to pursue making the managing of these enabled/disabled variables something that can be done or at least partially done in the UI --> Confluence Administration --> User Macros? Or at least the page could read the config and display what is enabled for macro.required.velocity.context.keys

Not all of our power users and confluence administrators that have access to define User Macros in the Web UI also necessarily have Server Administrator access to the config files e.g. <installation-directory>/bin/setenv.sh . 

Yes, some of this can be resolved through education of the admins. However. 

Even more ideally the entire list of theoretically usable variables per the documentation could be compared to what is actually enabled, and when macros are saved a detection could alert our power users / UI-only admins that their code includes a valid Velocity Context variable but an admin needs to enable it. This seems like it would save effort & support tickets on both the customer and Atlassian sides around "identifying the required context variables" which I have to suspect is coming up a lot and will continue coming up a lot as customers try to leverage business value from the User Macros feature. 

Workaround works

Hi bf8c41f5e8d5 and 812fda79c956 , thanks for reaching out to us. Can you please follow the workaround in the description section to add the new system property with the missing context variables? If you have trouble identifying the required context variables, please feel free to raise a support ticket or ask here with macro information. Thanks

Hi 624f66b7b4c0 , thanks for providing the macro information. I think you need to add one extra context variable `htmlUtil` to make it works.

"CATALINA_OPTS="-Dmacro.required.velocity.context.keys=generalUtil,bootstrap,action,pageManager,req,htmlUtil ${CATALINA_OPTS}"" 

Please have a try and let me know if it works. Thanks

@Jeffery Xie, thanks for your response and I am attaching the macro content and error on the Wiki page.

Error on the page : "$htmlUtil.htmlEncode($action.getUserFullName($username))"

Macro content :

1. 1. @noparams
      #usernameLink($content.getLastModifierName())

Same here 7.19.12

Same problem here: 7.19.11

Hi 624f66b7b4c0 , can you share the macro content or the error message when using the macro? so we can identify which specific context variable is required to be added to the system property. Thanks

I am facing same problem after upgrading Confluence to 7.19.12. One of the user macro to display "last user to modify" has stopped working.

#usernameLink($content.getLastModifierName())

I already added below parameter in configuration file which fixed other user macros but not "last user to modify" macro. Any suggestions on how to fix this?

"CATALINA_OPTS="-Dmacro.required.velocity.context.keys=generalUtil,bootstrap,action,pageManager,req ${CATALINA_OPTS}""

Hi 5717f2a94c18 ,  the variables "generalUtil" and "bootstrap" are default values set for the new system property "macro.required.velocity.context.keys" in the confluence application code, if you want to overwrite these default value of this system property, then you can set the desired value of it in setenv.sh. 

According to the editorial, the variables "generalUtil" and "bootstrap" are already set. In the binary of Atlassian version 7.19.12 I could not find the variables in the setenv.sh. Are they set in a hidden way? 

Thanks both, that worked!

Hi ca2db7ee88a8 ,

Thanks for reaching out to us and sorry for the issue. You need to add the extra velocity variable `action` into the system property with the two default ones. 

-Dmacro.required.velocity.context.keys=generalUtil,bootstrap,action 

You need to add "action":

-Dmacro.required.velocity.context.keys=generalUtil,bootstrap,action

Hi, 

We use a macro for the current time and it stopped working in 8.3.0:

$action.dateFormatter.formatGivenString("dd-MMM-yyyy hh:mm:ss", $content.currentDate)

In the macros it says "User Macro current-time may use context keys [* action *] which are not set in system property macro.required.velocity.context.keys. If the macro is not rendered as expected, try to manually add those keys into the system property."

I added the following and restarted Confluence but it didn't do anything.

CATALINA_OPTS="-Dmacro.required.velocity.context.keys=generalUtil,bootstrap ${CATALINA_OPTS}"

Do we need to do anything else?

hi bc9b412df521 ,

Thank you for reaching to us. Can you please have a look at the notes to the issue description above. And please review the purple noted section, and consider reviewing the allow listed values added to the instance.

Hello,
we have the same problem in version 8.3.2 Datacenter.

Regards,
Sven Hock

Hey f8c0b8f26c24,

No worries, I'm glad I could clear up some of the confusion.

Thanks for bearing with us on this one.

Regards,
James Ponting
Engineering Manager - Confluence Data Center

Thank you James, this update makes it very clear. I was indeed under the impression (judging from the description, fixed status, and few recent comments) that the workaround would no longer be required. Now with the updates this is all good and clear, thanks!

Hi All,

I just want to touch on some of the comments above.

87d239033716, c34449de0ab6, 66fb986bcf75 - I've added additional notes to the issue description above to help further explain this issue. Please review the purple noted section, and consider reviewing the allow listed values added to the instance.

f8c0b8f26c24 - Please see my previous comment on this issue.

The feedback provided (through various channels) that the initial change wasn't communicated sufficiently has been well heard. The change did not follow our normal processes for introducing potentially breaking changes, and as such did not include the appropriate communication. We've conducted a review of the incident and have corrected the miss that lead to this situation. Whilst I wasn't involved in the original situation, I do apologise for the inconvenience you've all faced.

Hopefully this helps clear up some of the confusion.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

Please be advised that Atlassian has changed the API in terms of which modules are allowed by default: User macro Velocity context

Thus the fix is browsing to /admin/usermacros.action, looking up the required modules from the warnings there and adding them to the system property macro.required.velocity.context.keys:

User Macro XXX may use context keys [ userAccessor, spaceManager ] which are not set in system property macro.required.velocity.context.keys. If the
macro is not rendered as expected, try to manually add those keys into the system property.

It was mentioned in James Pointings comment:

As the behaviour described in this issue is as intended, we'll not be "fixing it" per se. Instead we're investigating adding warnings to the User Macro management interfaces in Confluence Administration that will warn when a required context is used but not allow listed. Given the impact of this issue, I'll leave it open and track the addition of the warnings on this issue.

Does Atlassian just roll a "fix", which doesn't "fix", and then forget about it? Doesn't anybody read the comments whatsoever? This is by a long shot not the first time a "fix" was released, didn't work, and then the issue just sits in the backlog as "Closed" with no updates.

Yes, the bug is still present in version 7.19.9. The workaround works on this version.

Upgraded to 7.19.9 and bug is still present. Added the modules given in the workaround and it worked.

Thanks @ViktorS. 

I enabled only the modules in use and the macros are working again. 

@Augusto,  reg. "what should actually be enabled ?"

You can get all your custom code of all custom macros and search for "user-macro" in files "atlassian-plugin.xml". Entries would be like "<user-macro name="Foo" key="bar" hasBody="false">...". In macros entry check all mentioned variables like $action, $pageManager etc. In this way you get the list of all the keys you need to mention in JVM parameter.

upgrading to 7.19.9 does not Fix it. 

Only adding all modules described in the workaround: 

-Dmacro.required.velocity.context.keys=pageManager,spaceManager,attachmentManager,permissionHelper,userAccessor,i18n,dateFormatter,res,settingsManager,htmlUtil,generalUtil,bootstrap,containerContext,action,req,content 

but what should actually be enabled ?! 

A fix for this issue is available in Confluence Server and Data Center 8.3.0.
Upgrade now or check out the Release Notes to see what other issues are resolved.

To allow access to specific contexts, Confluence Administrators will need to explicitly allow permission to the contexts using the workaround described above.

This is a terrible workaround, as it involves restarting all your Confluence servers, therefore many will take the easiest option and simply make a one-time change to add all possible contexts to the JVM startup parameter.

If it's so important for security, there should be an Admin UI where we can control/monitor this properly.

Thanks for the contributions here. I think what would be helpful here is a specific workaround text string to put for the CATALINA_OPTS that renders the Confluence behavior around user macros exactly as it was before Atlassian "restricted the modules that can be used by User Macros out of the box" then, at the customer's option, we can add restrictions one-by-one as our security requirements dictate by modifying the CATALINA_OPTS. There seem to be several versions of what to put and I agree the description is incomplete - do not consider this issue closed. 

The above description is incomplete.

Workaround section:

• There is no i8n object as mentioned in the extended module list. This should be i18n instead. The same incorrect key is mentioned in the module example in CONFSERVER-83248

Notes section:

• The problematic res object mentioned in the notes is also registered for key response (when specified in the system property). Therefore this should also be removed when running into problems caused by the res key.

A fix for this issue is available in Confluence Server and Data Center 7.13.17.
Upgrade now or check out the Release Notes to see what other issues are resolved.

as far as I understood, this is not really a fix as "it's not a bug, it's a feature", and the "fix" is meant to only give a more understandable message telling you HAVE TO add
-Dmacro.required.velocity.context.keys=generalUtil,bootstrap,containerContext,action,req,content
in the java startup
Didn't test it myself as I already added this configuration so I don't even need this new version

Looks like the fix isn't working - upgraded on release 7.19.9 and it isn't working.

Unfortunately, the fix is not yet available. 

A fix for this issue is available in Confluence Server and Data Center 7.19.9.
Upgrade now or check out the Release Notes to see what other issues are resolved.

Fully agree with Carsten 2 points mentionned above !!!
Need clearer/fullfil release notes and a 8.x LTS soon

In which release was this security feature introduced? This is the 2nd time on short where we are caught by surprise that due to Security, we encounter a regression that was not easily spotted.

I was going through all release notes since our last SERVER version 7.20.3 (we just upgraded to 8.1.3) and could find no clear mentioning of this searching for the term "macro" (many other tickets but nothing shouting to be aware!)

How can we prepare for such thing if not clearly mentioned in release notes. maybe I missed, but where?

Lastly, you will fix this warning only for 8.3, not 8.1.x. Why is that? I wish there was a LTS on version 8 that we could fix on

thanks

Hey 873135d8df40,

Obviously I can't say definitively without investigating, but I would be surprised if the two were related.

Enabling/disabling of these modules shouldn't be affecting Collaborative Editing in any way, and the modules would still be needed regardless of whether Collaborative Editing was enabled or not.

There's a pretty easy regression test to run, you can identify a page that is impacted when Collaborative Editing is enabled, removed the module whitelist, restart Confluence, and confirm if the page is editable again. If you reproduce this consistently (i.e. more than once), then it there's potentially something there. I'd definitely recommend reaching out to our Support team as they can help investigate further

Let us know how you get on.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

As an update although I've not found hard evidence of a linkage between the issues, although the workaround got our macros working again this morning our Production system began encountering these issues for page edits/publishing:

https://confluence.atlassian.com/confkb/confluence-throws-the-editor-didn-t-load-this-time-error-when-trying-to-edit-a-page-840796639.html
https://community.atlassian.com/t5/Confluence-questions/Getting-error-quot-Unable-to-communicate-with-server-Saving-is/qaq-p/27058

Affected non-saving pages/drafts do not need / are not using the Page Properties Report Macro. 

Checked for & did not find a web application firewall (WAF). Restarted confluence and nginx and found nothing definitive in their logs, finally took the suggestion to (at least temporarily) disable Collaborative Editing and upon fresh page reloads edits/publishing began working with very fast saves again. And macros work. We probably do want to get CE fixed though. 

I applied the CATALINA OPTS posted by Hannes Medwed as the ones in the officially posted work around did not quite get our macros going. I'm going to try to reproduce everything in our dev/staging env but hope it is OK to ask here if there might be correlation between applying the workaround and impacts on Collaborative Editing? 

Hi f75f3e4197e6 , please refer to this documentation of configuring system properties according to your environment type.

With regards the workaround, which file needs to be modified?  What's the "java startup" file?  Thanks

Hey All,

I just wanted to provide some context on this issue for everyone.

The behaviour being seen with this issue is very much as intended. As part of improving the security stance of Confluence, we've added restrictions to what contexts (and by extension variables) can be used by default, as access to this tooling can expose a lot of information that may not be apparent to admins at first glance.

To allow access to specific contexts, Confluence Administrators will need to explicitly allow permission to the contexts using the workaround described above. As part of this issue, we also discovered that there's a potential for a null pointer exception when attempting to load contexts for a page where the context is null. This is what was occurring with the res context above. We're tracking this null pointer exception issue at CONFSERVER-83283: Page Properties Report Macro does not render User Macros when any default context item value is null

As the behaviour described in this issue is as intended, we'll not be "fixing it" per se. Instead we're investigating adding warnings to the User Macro management interfaces in Confluence Administration that will warn when a required context is used but not allow listed. Given the impact of this issue, I'll leave it open and track the addition of the warnings on this issue.

b484523b5d94, 873135d8df40 et all - I expect this will be backported. I'll update the ticket to reflect this (I've used our boiler plate above, I'll update it when I have potential release versions).

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

Also affected. In addition to Joel's question will this be applied to LTS 7.19.x?

Will this fix be applied to the LTS version 7.13.x?

It will be great to have that in setenv.sh by default.

We are using such user macros heavily and are affected as well.
For example evaluation of $action.dateFormatter.formatGivenString("yyyy-MM-dd", $content.lastModificationDate) is not working anymore.

Same problem here.

Also this one is not working anymore:

$action.dateFormatter.formatDateTime($content.getLastModificationDate())

Also the page module should be added as key if you want to use pages loaded with pageManager.loadPage

-Dmacro.required.velocity.context.keys=pageManager,page,spaceManager,attachmentManager,permissionHelper,userAccessor,i8n,dateFormatter,res,settingsManager,htmlUtil,generalUtil,bootstrap,containerContext,action,req,content