Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-37890

Page Properties Report is fetching a limited number of pages

      Summary

      Page Property Report Macros only fetches a limited number of pages.

      Environment

      Experienced in 6.3.1 while issues reported from previous versions listed in the "Affected Version"

      Steps to Reproduce

      How to reproduce method #1: Use the sample Space Export

      Get this XML space export 900pagesPPRmacro.zip which is a space that contains 900 pages containing Page Property macro with the labels "discussion"

      1. Load the page with the title "PP Report Macro with Number of items to display = 100" from the xml export
      2. Load the page with the title "PP Report Macro with Number of items to display = 10000" from the xml export
      How to reproduce method #2: create own space with 900 pages containing page-properties (any number more than 500):
      1. create a space with more than 500 (eg: 900) pages containing page-properties.
      2. create a page containing the page-properties-report macro and set the "number of objects to be shown" to 100 and 10000
        <p>
          <ac:structured-macro ac:macro-id="08027386-e7e9-4d92-8ad6-5168aa775020" ac:name="detailssummary" ac:schema-version="2">
            <ac:parameter ac:name="pageSize">100</ac:parameter>
            <ac:parameter ac:name="analytics-key">requirements</ac:parameter>
            <ac:parameter ac:name="cql">label = "product" and space = currentSpace()</ac:parameter>
          </ac:structured-macro>
        </p>
        

      Expected Results

      1. all 900 pages should be listed and the page should have 9 tabs appeared (100 result for every tab)
      2. all 900 pages should be displayed

      Actual Results

      1. only 5 tabs are listed in this page (100 pages on every tab = only displaying 500 pages)
      2. only 500 pages are listed
        • Results may vary from 500-550

      The following appears in atlassian-confluence.log:

      2017-07-16 16:59:58,699 WARN [pool-12-thread-2] [atlassian.confluence.event.MonitorableCallerRunsPolicy] rejectedExecution Queue Overflow happened: Runnable: [Invoker: com.atlassian.confluence.event.ConfluenceListenerHandlersConfiguration$TimingListenerHandler$1$1@16fe75dc; Event: com.atlassian.confluence.impl.content.render.xhtml.analytics.MarshallerMetricsAnalyticsEvent@bcfc4d6]; Executor: [java.util.concurrent.ThreadPoolExecutor@3569abb7[Running, pool size = 2, active threads = 2, queued tasks = 256, completed tasks = 30473]]
       -- url: /rest/masterdetail/1.0/detailssummary/lines | referer: http://localhost:8613/display/PPM/PP+Report+Macro+with+Number+of+items+to+display+%3D+10000 | traceId: 723d28b96b998853 | userName: admin
      
      1. Increasing http threads and database connection threads does not make any difference
      2. Disabling Analytics does not make any difference and the logs above still appears
      3. Increasing Page Property Report Caching does not make any difference

      Notes

      The issue is specifically caused by an issue in "fetching" the complete result. This is because when we change the Page Property Report macro configuration as "Reverse Sort" (and the pages are sorted based on modified date), the oldest pages now appears in the list while the recently created pages disappeared.

            [CONFSERVER-37890] Page Properties Report is fetching a limited number of pages

            If you're running the Confluence 7.4 Enterprise release, a fix for this issue is now available in Confluence 7.4.6, which you can find in the Download Archives.

            Oliver Shen added a comment - If you're running the Confluence 7.4 Enterprise release, a fix for this issue is now available in Confluence 7.4.6, which you can find in the Download Archives .

            Alex K added a comment -

            Hi f56fabbcc46c, 7.4.6 will be released within a couple of weeks.

            Alex K added a comment - Hi f56fabbcc46c , 7.4.6 will be released within a couple of weeks.

            Anton added a comment -

            Are you going to do the release 7.4.6?

            Anton added a comment - Are you going to do the release 7.4.6?

            A fix for this issue is available to Server and Data Center customers in Confluence 7.7.2
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            Xinyi Xu (Inactive) added a comment - A fix for this issue is available to Server and Data Center customers in Confluence 7.7.2 Upgrade now or check out the Release Notes  to see what other issues are resolved.

            Hello James,

            I am glad to hear that. A fix for V7.4 would make me happy, our strategy is to walk along with Enterprise Releases.

            For internal properties, we have a work-around. For Comala I will contact Comala.

            Thanks, Manfred Wiplinger

            Manfred Wiplinger added a comment - Hello James, I am glad to hear that. A fix for V7.4 would make me happy, our strategy is to walk along with Enterprise Releases. For internal properties, we have a work-around. For Comala I will contact Comala. Thanks, Manfred Wiplinger

            Hey Manfred,

            Thanks very much for the additional information.

            We have a fix pending for CONFSERVER-56148: Intermittent NPE errors when using one or more Page Property Macro or any other macro, so hopefully that will resolve that issue entirely. I'll flag the issue for review for backporting to an Enterprise Release as well.

            With regards to adding internal properties to reports, I'm not sure if that falls within the scope of the work we're doing currently (it changes the way the report works, from importing macro content to a larger page object scope), but I'll definitely pass it on to Makisa and the team.

            With regards to the Comala Document Mangement metadata, that's something the Comala team may be able to implement. If the current design approach of pulling data from the Page Properties macro continues, they may be able to create a macro that pulls that information into the Page Properties macro table. I'd suggest following up with their support team, their details are available at Comala Document Management - Support - Atlassian Marketplace.

            Thanks again for the insight into how you're using the Page Property Report. If we end up with a feature request around modified date etc, I'll update this reply with a link.

            Regards,
            James Ponting
            Premier Support Engineer

            James Ponting added a comment - Hey Manfred, Thanks very much for the additional information. We have a fix pending for CONFSERVER-56148: Intermittent NPE errors when using one or more Page Property Macro or any other macro , so hopefully that will resolve that issue entirely. I'll flag the issue for review for backporting to an Enterprise Release as well. With regards to adding internal properties to reports, I'm not sure if that falls within the scope of the work we're doing currently (it changes the way the report works, from importing macro content to a larger page object scope), but I'll definitely pass it on to Makisa and the team. With regards to the Comala Document Mangement metadata, that's something the Comala team may be able to implement. If the current design approach of pulling data from the Page Properties macro continues, they may be able to create a macro that pulls that information into the Page Properties macro table. I'd suggest following up with their support team, their details are available at Comala Document Management - Support - Atlassian Marketplace . Thanks again for the insight into how you're using the Page Property Report. If we end up with a feature request around modified date etc, I'll update this reply with a link. Regards, James Ponting Premier Support Engineer

            hello James,

            Our top use case is "on demand" report of the state of our technical documentation. We list documents with label "products" across multiple (about 15) spaces. The report lists about 3000 products. Today we provide one report per space to reduce the size. That reduces the appearance of exceptions (CONFSERVER-56148).

            The reports contains text properties like status (evaluation, building, running), people (technical responsible, responsible for documentation), date (last review date). Not more than 10 properties within one report, else it becomes to broad.

            The most wanted feature is to add some internal properties (eg. modified by) directly within a report. Today we put this internal properties into page properties on every page. 

            And, I would like to add properties from "Comala document management" (eg. state, expiration date) too.

             

            Manfred Wiplinger added a comment - hello James, Our top use case is "on demand" report of the state of our technical documentation. We list documents with label "products" across multiple (about 15) spaces. The report lists about 3000 products. Today we provide one report per space to reduce the size. That reduces the appearance of exceptions ( CONFSERVER-56148 ). The reports contains text properties like status (evaluation, building, running), people (technical responsible, responsible for documentation), date (last review date). Not more than 10 properties within one report, else it becomes to broad. The most wanted feature is to add some internal properties (eg. modified by) directly within a report. Today we put this internal properties into page properties on every page.  And, I would like to add properties from "Comala document management" (eg. state, expiration date) too.  

            Hi All,

            I'm currently working with mappleton@atlassian.com and the team working on this issue.

            We believe we've accounted for most use cases in the work we're planning, but we want to ensure we have visibility of your teams needs.

            To that end, we have a couple of questions about your largest reports if you're able to help us out with some information (understanding that this is a public forum).

            Can you please tell us

            • The use case of reporting (Used for compliance reporting daily, weekly, monthly, annually etc etc)
            • The size (rows) of your biggest report currently (if it weren't trimmed)
            • The number of properties contained within your biggest reports
            • The types of properties being used (i.e. text, status lozenge, date object etc etc)
            • Any additional issues you're encountering with Page Properties Reports outside of this issue and CONFSERVER-56148

            If anyone had anything to add, please let us know as now is definitely the time

            Thanks,
            James Ponting
            Premier Support Engineer

            James Ponting added a comment - Hi All, I'm currently working with mappleton@atlassian.com and the team working on this issue. We believe we've accounted for most use cases in the work we're planning, but we want to ensure we have visibility of your teams needs. To that end, we have a couple of questions about your largest reports if you're able to help us out with some information (understanding that this is a public forum). Can you please tell us The use case of reporting (Used for compliance reporting daily, weekly, monthly, annually etc etc) The size (rows) of your biggest report currently (if it weren't trimmed) The number of properties contained within your biggest reports The types of properties being used (i.e. text, status lozenge, date object etc etc) Any additional issues you're encountering with Page Properties Reports outside of this issue and CONFSERVER-56148 If anyone had anything to add, please let us know as now is definitely the time Thanks, James Ponting Premier Support Engineer

            Hi grzwip thanks for your comment; we'll take a look at CONFSERVER-56148.

            Makisa Appleton added a comment - Hi grzwip thanks for your comment; we'll take a look at CONFSERVER-56148 .

            Hi Makisa,

            my bigger problem is, that long lists of pages show errors (HibernateSystemException), sometimes. Sometimes reload of the report page shows proper reports.

            Seems to be this Issue https://jira.atlassian.com/browse/CONFSERVER-56148

             

            Causeorg.springframework.orm.hibernate.HibernateSystemException: Found shared references to a collection: com.atlassian.confluence.core.ContentEntityObject.outgoingLinks; nested exception is net.sf.hibernate.HibernateException: Found shared references to a collection: com.atlassian.confluence.core.ContentEntityObject.outgoingLinks
                at org.springframework.orm.hibernate.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:597)caused by: net.sf.hibernate.HibernateException: Found shared references to a collection: com.atlassian.confluence.core.ContentEntityObject.outgoingLinks
                at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:147)

            Manfred Wiplinger added a comment - Hi Makisa, my bigger problem is, that long lists of pages show errors (HibernateSystemException), sometimes. Sometimes reload of the report page shows proper reports. Seems to be this Issue  https://jira.atlassian.com/browse/CONFSERVER-56148   Cause org.springframework.orm.hibernate.HibernateSystemException: Found shared references to a collection: com.atlassian.confluence.core.ContentEntityObject.outgoingLinks; nested exception is net.sf.hibernate.HibernateException: Found shared references to a collection: com.atlassian.confluence.core.ContentEntityObject.outgoingLinks     at org.springframework.orm.hibernate.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:597) caused by: net.sf.hibernate.HibernateException: Found shared references to a collection: com.atlassian.confluence.core.ContentEntityObject.outgoingLinks     at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:147)

              aknight@atlassian.com Alex K
              25ecd6effc02 georg
              Affected customers:
              87 This affects my team
              Watchers:
              99 Start watching this issue

                Created:
                Updated:
                Resolved: