As of the time of this writing, Custom fields at the system level and Next Gen custom fields have an extreme level of collision with each-other. Search is unable to clearly understand which custom field is intended unless you use the extremely ambiguous cf_id format in JQL searching. This causes the JQL searches to be extremely ugly, or requires teams to coordinate on their use of custom fields.

In order to resolve these issues, I would like to propose the use of Namespaces for custom fields. The idea would be to allow users to refer to custom fields with the following syntax:

projectkey:customfieldname

An example would be like:
ABC:EstimatedCost
or
"BAR:Story Points"
This would allow you to specify from which project you want to get the custom field, and would allow macros and other systems to identify custom fields from Next Gen projects more clearly.

For the globally defined custom fields for Classic projects, you can denote them specifically with this syntax:
:EstimatedCost
or
":Story Points"

This would allow you to specifically refer to the classic custom fields without potentially clashing with the namespace of any projects by key. This also opens the door to the third option of not using the namespace ":" designator.

By using the syntax of today, you could potentially implement "smart masking" where only values that are NOT NULL/ NOT EMPTY are returned allowing for values to be combined into a single column on reports for when the name is the same.

This would make the returned information from searches and other systems (such as Confluence Jira Issue Macro) more intuitive and intelligent while also providing more specificity and not requiring a fundamental restructure of the database systems.