Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-95272

REST API pagination (eg, /rest/api/space) returns more data than available

      Issue Summary

      This issue relates to general paginated results.

      Requesting data from an endpoint such as /rest/api/space or rest/api/content causes Confluence to return more data than available.

      This is reproducible on Data Center: yes

      Steps to Reproduce

      1. Request /rest/api/space to collect all of the spaces available without using ?expand parameter;
      2. Verify that Confluence returns the exact amount of spaces that exist in the instance;
      3. Now add the ?expand parameter, for example:
        1. https://fqdn/rest/api/space?expand=description.plain&limit=1&start=999

      Expected Results

      Using ?expand produces the same array as without it, except that it expands the required fields.

      Actual Results

      Using ?expand causes Confluence to keep returning data past the available amount, which breaks apps that loop through the resulting array.

      Workaround

      Currently there is no known workaround for this behavior. But if you have Comala Document management app and noticed that the REST calls are related to Comala like below:

      https://ConfluecneBaseURL/rest/cw/2/report/search?cql=awphasworkflow%3Dtrue%20AND%20space%20IN%20(%22TE%22)%20AND%20state%20IN%20 

      Then Comala Tech has shared few steps to lower the CPU usage. Basically, number of items to be displayed in the report should be 11 or less. Below are the steps recommended by Comala tech.

       

      1. Go to a page where you have the Document States Report macro failing
      2. Edit the page
      3. Edit the macro and modify the value of “Number of items to display” and set it to “11”
      4. Save the page
      5. Check if the report is rendered successfully

            [CONFSERVER-95272] REST API pagination (eg, /rest/api/space) returns more data than available

            A fix for this issue is available in Confluence Server and Data Center 8.5.9.
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            James Whitehead added a comment - A fix for this issue is available in Confluence Server and Data Center 8.5.9. Upgrade now or check out the Release Notes to see what other issues are resolved.

            Could you also share the information when the fix version for the latest LTS (8.5) will be released?

            Ganna Kucher added a comment - Could you also share the information when the fix version for the latest LTS (8.5) will be released?

            A fix for this issue is available in Confluence Data Center 8.9.1.
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            James Whitehead added a comment - A fix for this issue is available in Confluence Data Center 8.9.1. Upgrade now or check out the Release Notes to see what other issues are resolved.

            A fix for this issue is available in Confluence Server and Data Center 7.19.22.
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            James Whitehead added a comment - A fix for this issue is available in Confluence Server and Data Center 7.19.22. Upgrade now or check out the Release Notes to see what other issues are resolved.

            Pagination works incorrectly when executing a request like:

            /rest/api/content?spaceKey=ADRADR&start=0&status=current&expand=body.storage

            start=0 and start=100 returns the same set of values. Next, all values that are multiples of 100 return the same result.

            If you delete expand=body.storage, pagination works correctly.

            Ravil Sabirov added a comment - Pagination works incorrectly when executing a request like: /rest/api/content?spaceKey=ADRADR&start=0&status=current&expand=body.storage start=0 and start=100 returns the same set of values. Next, all values that are multiples of 100 return the same result. If you delete expand=body.storage, pagination works correctly.

            We are facing a similar issue on Confluence 8.5.6 with Comala Document Management 7.2.3. We are seeing stuck threads for the following API call. It times out after 30 seconds and gives an error.  We have a support ticket open with the Appfire/Comala team.

            Karen.Mustain added a comment - We are facing a similar issue on Confluence 8.5.6 with Comala Document Management 7.2.3. We are seeing stuck threads for the following API call. It times out after 30 seconds and gives an error.  We have a support ticket open with the Appfire/Comala team.

            Something similar seems to be happening on the following REST API without adding any expand parameter.

            https://<baseurl>/rest/api/space/<spacekey>/content?limit=100&start=9000

            The space in question only has ~3900 Pages, but keeps returning well over 9000.
            The "next" link in the json keeps getting populated even after asking for more than the existing number of pages.

            ...
                "start": 40000,
                "limit": 100,
                "size": 100,
                "_links": {
                  "self": "https://<baseurl>/rest/api/space/Productivity/content/page",
                  "next": "/rest/api/space/Productivity/content/page?limit=100&start=40100",
                  "prev": "/rest/api/space/Productivity/content/page?limit=100&start=39900"
                }
            ...
            

            Sophie Schiller added a comment - Something similar seems to be happening on the following REST API without adding any expand parameter. https://<baseurl>/rest/api/space/<spacekey>/content?limit=100&start=9000 The space in question only has ~3900 Pages, but keeps returning well over 9000. The "next" link in the json keeps getting populated even after asking for more than the existing number of pages. ... "start" : 40000, "limit" : 100, "size" : 100, "_links" : { "self" : "https: //<baseurl>/ rest /api/space/Productivity/content/page" , "next" : "/ rest /api/space/Productivity/content/page?limit=100&start=40100" , "prev" : "/ rest /api/space/Productivity/content/page?limit=100&start=39900" } ...

            1. Have developers reviewed the code changes made in this area for 8.9.0 and 8.5.8?
            2. Once the system gets into this state, it is 100% repeatable including after restarts.
            3. At least one customer has reported the problem only a few days after upgrading to 8.5.8.
            4. We can reproduce the problem on multiple 8.5.8 and 8.9.0 instances after running various integration tests including different tests on different instances. It seems to take a fair bit of activity to flip the system into the bad behavior. We have an integration test pipeline that ALWAYS produces a corrupted instance in about 10 minutes.

            Bob Swift {Appfire} added a comment - Have developers reviewed the code changes made in this area for 8.9.0 and 8.5.8? Once the system gets into this state, it is 100% repeatable including after restarts. At least one customer has reported the problem only a few days after upgrading to 8.5.8. We can reproduce the problem on multiple 8.5.8 and 8.9.0 instances after running various integration tests including different tests on different instances. It seems to take a fair bit of activity to flip the system into the bad behavior. We have an integration test pipeline that ALWAYS produces a corrupted instance in about 10 minutes.

              24ee36512d14 Sumitra Sahu (Inactive)
              resteves Rodrigo Heinzen
              Affected customers:
              45 This affects my team
              Watchers:
              64 Start watching this issue

                Created:
                Updated:
                Resolved: