Issue Summary

In Jira Data Center 11.3.x, a number of public APIs are provided which allow consumers to trigger the customfield indexers and searchers caches to be reset. If the callers of these do not trigger an expected cache invalidation events e.g. {{ClearCacheEvent, CustomFieldCreatedEvent, CustomFieldUpdatedEvent, }}etc, then the fields cache (used to index field values) will become stale.

When this happens, reindexing or issue operations may fail with an error similar to:

com.atlassian.jira.search.exception.InvalidDocumentFieldException: Document contains invalid fields [customfield_XXXXX]

This can affect code paths that add, update, or refresh custom field/search metadata through APIs such as, but not limited to:

• SearchHandlerManager
• IssueSearcherManager
• FieldManager
• CustomFieldManager

and their respective refresh methods.

Steps to Reproduce

A consistently reliable reproduction is not yet available.

A possible reproduction pattern is:

1. Add or update a custom field through an app, script, integration, import, or Jira code path.
2. Ensure the operation modifies field/search metadata through APIs such as SearchHandlerManager, IssueSearcherManager, FieldManager, or CustomFieldManager.
3. Ensure the operation does not emit a ClearCacheEvent, CustomFieldCreatedEvent, CustomFieldUpdatedEvent, or equivalent cache invalidation event.
4. Ensure the operation performs a refresh() call on the API used in 2.
5. Perform a full reindex or an issue operation that triggers issue indexing.

Expected Results

Jira refreshes the relevant custom field search metadata and field schema cache consistently.

Reindexing and issue operations complete successfully.

Actual Results

The field schema cache may remain stale.

Reindexing or issue operations may fail with:

com.atlassian.jira.search.exception.InvalidDocumentFieldException: Document contains invalid fields [customfield_XXXXX]

Example stack trace:

java.lang.RuntimeException: com.atlassian.jira.search.exception.InvalidDocumentFieldException: Document contains invalid fields [customfield_10000]
    at com.atlassian.jira.index.DefaultIndex$Failure.<init>(DefaultIndex.java:100)
    at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:959)
    at com.atlassian.jira.issue.index.DefaultIssueIndexer.reindexIssues(DefaultIssueIndexer.java:427)
    ...
Caused by: com.atlassian.jira.search.exception.InvalidDocumentFieldException: Document contains invalid fields [customfield_10000]
    at com.atlassian.jira.search.field.SchemaValidatedFieldValueCollector.validate(SchemaValidatedFieldValueCollector.java:38)
    at com.atlassian.jira.search.issue.index.indexers.impl.SortableTextCustomFieldIndexer.indexFieldsWithVisibility(SortableTextCustomFieldIndexer.java:83)
    ...

Customer Impact

When this problem occurs, customers may be unable to complete a full reindex or perform issue operations that trigger indexing.

Possible symptoms include:

• full reindex failures;
• issue transition failures;
• issue edit or update failures.

Workaround

A temporary workaround is to force Jira to refresh custom field/search metadata.

One way to do this is:

1. Go to Administration > Issues > Custom fields.
2. Select any custom field.
3. Choose Edit details.
4. Save the field without making changes.
5. Retry the failed reindex or issue operation.

This workaround works because saving a custom field through the UI triggers the relevant custom field update/cache invalidation path (ClearCacheEvent).

The workaround may need to be repeated after restart or after the affected field/search metadata becomes stale again.

Notes:

This issue was identified while resolving JRASERVER-79420.

The underlying gap is that Jira's field schema cache depends on custom field/search metadata, but not all code paths that add, update, or refresh that metadata emit a cache invalidation event that causes the field schema cache to refresh.

A previous investigation also showed that using broader cache reset events can restore correctness, but may introduce unnecessary cache invalidation and performance regressions. Jira needs a narrower and reliable invalidation mechanism so field schema cache state stays consistent without excessive refreshes.

A number of callers of the refresh() methods have been identified within Jira, all of which predate Jira 11.3.x. Our intention is possibly minimise the internal callers of these methods or look at other strategies to reduce the number of times the fields cache needs to respond to invalidation events, when originating from these refreshes.

Related Links:

• JRASERVER-79420: https://jira.atlassian.com/browse/JRASERVER-79420