OpenSearch bulk indexing payload can exceed the configured http.max_content_length limit

XMLWordPrintable

    • 1
    • Severity 3 - Minor

      Issue Summary

      When Confluence is configured to use OpenSearch as search engine instead of using the local Lucene, the final serialised OpenSearch document size to be sent to OpenSearch (/_bulk endpoint) may exceed the http.max_content_length limit.
      Resulting in OpenSearch rejecting with 413 Request Entity Too Large.

      Steps to Reproduce

      Environment: this is tested in Confluence 10.2.14, but should impact any Confluence versions that support OpenSearch released before the creation of this ticket.

      1. Configure Confluence to use OpenSearch as search engine
      2. Enable the logging package below as DEBUG level:
        • com.atlassian.confluence.plugins.opensearch.bulk
      3. Upload an attachment large enough to produce final serialised OpenSearch document to be above 100MB

      Expected Results

      As OpenSearch is an external service where the http.max_content_length can be vastly different depending on the hosting environment, Confluence should allow admins to configure the maximum payload to be sent to OpenSearch.

      Actual Results

      Confluence admin is not able to precisely control the limit of payload size to be sent to OpenSearch. When an Index Document goes beyond the OpenSearch limit, the logs below will be observed:

      atlassian-confluence-index.log
      2026-09-10 01:14:03,488 WARN [Indexer: 1] [confluence.internal.index.DefaultBatchIndexer] tryIndex Error getting document from searchable
      org.opensearch.client.transport.httpclient5.ResponseException: method [POST], host [https://<OpenSearch_URL>], URI [/_bulk], status line [HTTP/1.1 413 Request Entity Too Large]
      
      atlassian-confluence.log
      2026-09-10 01:14:03,174 INFO [Indexer: 1] [plugins.opensearch.bulk.MaxRamBulkOperationBuffer] flush Flushing bulk operation (max RAM reached), size (MB): 198.939
      2026-09-10 01:14:03,488 WARN [Indexer: 1] [plugins.opensearch.spring.OpenSearchRetryRegistryConfig] lambda$onEntryAdded$4 Operation bulk, 0 attempts, ignored error
      org.opensearch.client.transport.httpclient5.ResponseException: method [POST], host [https://<OpenSearch_URL>], URI [/_bulk], status line [HTTP/1.1 413 Request Entity Too Large]
      	at org.opensearch.client.transport.httpclient5.ApacheHttpClient5Transport.extractAndWrapCause(ApacheHttpClient5Transport.java:1147)
      	at org.opensearch.client.transport.httpclient5.ApacheHttpClient5Transport.performRequest(ApacheHttpClient5Transport.java:160)
      ...
      Caused by: org.opensearch.client.transport.httpclient5.ResponseException: method [POST], host [https://<OpenSearch_URL>], URI [/_bulk], status line [HTTP/1.1 413 Request Entity Too Large]
      	at org.opensearch.client.transport.httpclient5.ApacheHttpClient5Transport.convertResponse(ApacheHttpClient5Transport.java:322)
      	at org.opensearch.client.transport.httpclient5.ApacheHttpClient5Transport.access$400(ApacheHttpClient5Transport.java:107)
      	at org.opensearch.client.transport.httpclient5.ApacheHttpClient5Transport$1.completed(ApacheHttpClient5Transport.java:231)
      ...
      

      Extra Information

      • Index Rebuild and Large Attachment is used as a sample logs above, but this behavior is not limited to these conbination, but also include other content types as well as incremental indexing.
      • The original attachment size, the extracted text size, and the final serialised OpenSearch document size are distinctively different.
        • The atlassian.indexing.attachment.maxsize can be used to limit the size where Confluence would extract for the attachments, but this bug is related to the final serialised OpenSearch document size instead
        • Read more about How attachments are indexed

      Workaround

      Confluence 9.2.26 and later, and 10.2.21 (unrelease) and later, provide a dark feature that applies both a document-count limit and a RAM buffer limit during incremental indexing.

      1. Log in to Confluence as a System Administrator.
      2. Go to <CONFLUENCE_BASE_URL>/admin/darkfeatures.action.
      3. In the Site Dark Features field, enter confluence.opensearch.incremental.bulk.ram.buffer.enable.
      4. Select Submit.

              Assignee:
              Gabriel Weyer
              Reporter:
              Damien Tan
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: