Improve performance when deleting comments or blog posts in spaces with many watchers

XMLWordPrintable

    • 1
    • 7

      Summary

      Deleting a comment from a page with many watchers, or many space watchers (100+) is slow and can take more than 60 seconds.

      Environment

      • Confluence 6.4.3

      Steps to Reproduce

      1. Set up Confluence 6.4.3 (or latest)
      2. Use the Atlassian Populator to create 1000 users, 100 at a time.
      3. Go to Confluence and create a Space and watch that Space.
      4. Go to the database, and get the spaceid of the Space you're working with, e.g.
        select spaceid from spaces where spacekey = 'BUG';
        

        Let's say it's 12345

      5. Update the notifications table so all the users watch the sample Space
        insert into notifications (notificationid, spaceid, username, creator, creationdate, lastmodifier, lastmoddate, digest, network) (select floor(random()*1000000), 12345, user_key, user_key, current_timestamp, user_key, current_timestamp, false, false from user_mapping);
        

        You may have to run this several times before you get 1000 unique random numbers. (If you know a better way to grab an id, please share.)

      6. Go to General Configuration > Cache Management, scroll down and click Flush all
      7. Create a page, add a comment (this is pretty quick)
      8. Delete the comment (this is pretty slow)

      Expected Results

      The comment is deleted quickly

      Actual Results

      The below exception is thrown in the atlassian-confluence.log file:

      2017-11-02 18:28:01,560 WARN [http-nio-8090-exec-6] [confluence.util.profiling.DurationThresholdWarningTimingHelperFactory] logMessage Execution time for publishing event com.atlassian.confluence.event.events.content.comment.CommentRemoveEvent[source=com.atlassian.confluence.pages.DefaultCommentManager@149e17af] took 33451 ms (warning threshold is 5000 ms)
       -- url: /pages/removecomment.action | page: 2228571 | traceId: 71638ddfefef7f55 | userName: admin | referer: http://confluence.mycompany.com:8090/display/BUG/CONFSERVER-54140 | action: removecomment
      

      and catalina.out will also have similar warnings like this

      25-Oct-2017 00:27:55.595 WARNING [ContainerBackgroundProcessor[StandardEngine[Standalone]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread "http-nio-8150-exec-50" (id=263) has been active for 61,319 milliseconds (since 10/25/17 12:26 AM) to serve the same request for https://confluence.mycompany.com/pages/removecomment.action?commentId=133300287&pageId=120587236&atl_token=71472d2d5b654afc06a736af45a98f718cd3aa50&confirm=yes and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 1 thread(s) in total that are monitored by this Valve and may be stuck.
       java.lang.Throwable
      

      Notes

      The slow code is going through a synchronous call to send the notifications. Thread dumps show most of the time is spent in

      at com.atlassian.confluence.mail.notification.listeners.CommentNotificationsListener.handleCommentRemove(CommentNotificationsListener.java:71)
      

      Workaround

      There is no workaround.

            Assignee:
            Unassigned
            Reporter:
            James Richards
            Votes:
            5 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: