Uploaded image for project: 'Jira Platform Cloud'
  1. Jira Platform Cloud
  2. JRACLOUD-59980

JQL filter for Webhooks doesn't work correctly when "Comment", "Worklog", "Attachment" or "IssueLink" related events are fired

      NOTE: This bug report is for JIRA Cloud. Using JIRA Server? See the corresponding bug report.

      Atlassian Update – 22 January 2018

      Hi everyone,

      Thank you for voting and commenting on this request. After taking into account actual usage of different webhook events and potential workarounds for restricting them, we decided to action in the following ways:

      1. We are splitting this collective request into separate issues - each for individual webhook event - in order to make it more clear what is the status of JQL support for each of webhook events respectively, and to give you opportunity to express demand for specific improvements. From now on you can refer to the following issues for JQL filtering of:
        1. Comment webhooks - JRACLOUD-68235
        2. Attachment webhooks - JRACLOUD-68594
        3. Worklog webhooks - JRACLOUD-68595
      2. Since December 2017, JQL filtering is available for Comment webhooks. For the remaining webhook events, it is still possible to restrict them by substituting variables in the webhook URL.
      3. We have updated Jira Cloud Webhooks documentation to make it more clear that JQL filtering is only supported for a subset of webhook events, and what those events are.
      4. As of January 2018 JQL filtering of Attachment, Worklog and Issue Link webhooks (as well as others not mentioned as JQL enabled in the documentation) is not supported, and that works by design. Therefore, adding such functionality is treaded as improvement suggestion in the newly created issues.
      5. At this moment we do not have plans to add JQL filtering for the remaining webhook events, but we can revisit that decision in the future.
      6. As a consequence of splitting the issue, and because the main use case was addressed, we’re closing as Fixed.

      We understand that your expectations expressed in this issue have not been fully addressed, but after balancing the value of competing priorities we’ve determined that focusing on Comment webhooks would provide the biggest value for Jira Cloud app developers. To that end, I hope you appreciate the transparency of this update.

      Regards,
      Eve Stankiewicz
      Product Manager, Jira Cloud Platform

      Original request description:

      Summary

      JQL filter for Webhooks doesn't work correctly when the following related events are fired:

      • Comment
      • Worklog
      • attachment
      • issuelink

      Steps to Reproduce

      1. Create an webhook as follows
        • Issue related events
          • JQL: project = <PROJECTKEY>
          • Issue: created, updated
          • Issue link: created deleted
          • Comment: created, updated
          • Worklog: created,updated
          • Attachment: created, deleted
      2. Make a comment, log work, upload an attachment or link an issue in a project aside from the one which is specified with the JQL

      Expected Results

      The webhook shouldn't be fired.

      Actual Results

      The webhook was fired.

      Notes

      When it comes to NOT "Comment" or "Worklog" related events (like creating issue, updating issue),
      the JQL filters the events correctly.

        1. webhook.png
          webhook.png
          323 kB
        2. webhooks.png
          webhooks.png
          27 kB

          Form Name

            [JRACLOUD-59980] JQL filter for Webhooks doesn't work correctly when "Comment", "Worklog", "Attachment" or "IssueLink" related events are fired

            imbyungjun added a comment -

            Does it fixed?? I am still suffering from this issue in Jira 7.10.1

            imbyungjun added a comment - Does it fixed?? I am still suffering from this issue in Jira 7.10.1

            'Fix Version' is blank. When would this fix be available?

            Thanks.

            Gagan Bhatnagar added a comment - 'Fix Version' is blank. When would this fix be available? Thanks.

            Is there a JIRA Server equivalent of that trello roadmap page?

            Nathan Neulinger added a comment - Is there a JIRA Server equivalent of that trello roadmap page?

            nneul@neulinger.org, at this moment we're only rolling this change for Jira Cloud. In case similar change will be going to Jira Server, there will be a separate update in the JRASERVER-59980.

            We agree that filtering would be useful for Worklog and Attachment events as well, however we decided to go first with Comment events as a priority, because their volume and impact on apps is much greater than that of worklogs and attachments. But it is likely that filtering will be added in the future to remaining webhook events as well. And until that happens, I encourage you to check the public roadmap for Jira Cloud Ecosystem to get some visibility into the investments that we're prioritising above filtering of webhooks.

            Eve (Inactive) added a comment - nneul@neulinger.org , at this moment we're only rolling this change for Jira Cloud. In case similar change will be going to Jira Server, there will be a separate update in the JRASERVER-59980 . We agree that filtering would be useful for Worklog and Attachment events as well, however we decided to go first with Comment events as a priority, because their volume and impact on apps is much greater than that of worklogs and attachments. But it is likely that filtering will be added in the future to remaining webhook events as well. And until that happens, I encourage you to check the public roadmap for Jira Cloud Ecosystem to get some visibility into the investments that we're prioritising above filtering of webhooks.

            Are you also addressing it in JRASERVER-59980 ?

            @istankiewicz Can you give an explanation as to why you would NOT also apply these to the other events?

            I can understand the edge case for issue linking (needing to apply to BOTH issues independently) - but, I can't think of any reason why you'd ever want the event to fire for a worklog or attachment and ignore the JQL filter. Do you have a case where you'd want that?

            Nathan Neulinger added a comment - Are you also addressing it in JRASERVER-59980 ? @istankiewicz Can you give an explanation as to why you would NOT also apply these to the other events? I can understand the edge case for issue linking (needing to apply to BOTH issues independently) - but, I can't think of any reason why you'd ever want the event to fire for a worklog or attachment and ignore the JQL filter. Do you have a case where you'd want that?

            Hi all, I'd like to give you heads up that we're working on adding JQL filtering for Comment webhook events.

            Refer to Change notice - JQL filters applied to comment webhooks in Jira Cloud for details.

            Once the improvement is fully rolled out, we would remove references to Comment events from this issue. Note, that currently there is no plan to apply JQL filtering for remaining webhook events.

            Sincerely,
            Eve Stankiewicz
            Product Manager for Jira Cloud Ecosystem

            Eve (Inactive) added a comment - Hi all, I'd like to give you heads up that we're working on adding JQL filtering for Comment webhook events. Refer to Change notice - JQL filters applied to comment webhooks in Jira Cloud for details. Once the improvement is fully rolled out, we would remove references to Comment events from this issue. Note, that currently there is no plan to apply JQL filtering for remaining webhook events. Sincerely, Eve Stankiewicz Product Manager for Jira Cloud Ecosystem

            I took a look at the sources and it looks like fixing this may be simple - some notes below.
            I may be misinterpreting here, but it looks like just getting the comment event handlers changed
            to the 4 argument creator functions would address the lack of filtering.

            jira-project/jira-components/jira-plugins/jira-webhooks-plugin/src/main/java/com/atlassian/jira/plugins/webhooks

            registration/RegisteredWebHookEventFactory.java:
            
                private RegisteredWebHookEvent event(final String id, final String namei18nKey, final Class<?> eventClass) {
                    return RegisteredWebHookEvent
                            .withId(id)
                            .andDisplayName(namei18nKey)
                            .firedWhen(eventClass)
                            .isMatchedBy(EventMatchers.ALWAYS_TRUE);
                }
            
                private <T> RegisteredWebHookEvent event(String id, String displayName, Class<T> eventClass, EventMatcher<? super T> matcher) {
                    return RegisteredWebHookEvent
                            .withId(id)
                            .andDisplayName(displayName)
                            .firedWhen(eventClass)
                            .isMatchedBy(matcher);
                }
            

            the call to set up the event handler for issue events calls event() using the 4 argument form:

                    WebHookEventGroup issueGroup = WebHookEventGroup.builder()
                            .nameI18nKey("webhook.group.issue")
                            .addEvent(event("jira:issue_created", "webhook.group.issue.created", IssueEvent.class, Sets.newHashSet(EventType.ISSUE_CREATED_ID)))
                            .addEvent(event("jira:issue_updated", "webhook.group.issue.updated", IssueEvent.class, Sets.newHashSet(EventType.ISSUE_ASSIGNED_ID,
                                    EventType.ISSUE_CLOSED_ID, EventType.ISSUE_REOPENED_ID, EventType.ISSUE_RESOLVED_ID,
                                    EventType.ISSUE_COMMENT_EDITED_ID, EventType.ISSUE_COMMENTED_ID, EventType.ISSUE_MOVED_ID,
                                    EventType.ISSUE_UPDATED_ID, EventType.ISSUE_WORKSTARTED_ID, EventType.ISSUE_WORKSTOPPED_ID,
                                    EventType.ISSUE_GENERICEVENT_ID, EventType.ISSUE_COMMENT_DELETED_ID)))
                            .addEvent(event("jira:issue_deleted", "webhook.group.issue.deleted", IssuePreDeleteEvent.class, Collections.<Long>emptySet()))
                            .addEvent(event("jira:worklog_updated", "webhook.group.issue.worklog.changed", IssueEvent.class, Sets.newHashSet(EventType.ISSUE_WORKLOG_UPDATED_ID,
                                    EventType.ISSUE_WORKLOG_DELETED_ID, EventType.ISSUE_WORKLOGGED_ID)))
                            .build();
            

            where the comment and worklog handlers use the 3 argument form with the corresponding "EventMatches.ALWAYS_TRUE":

                    WebHookEventGroup commentGroup = WebHookEventGroup.builder()
                            .nameI18nKey("webhook.group.comment")
                            .addEvent(event("comment_created", "webhook.group.comment.created", CommentCreatedEvent.class))
                            .addEvent(event("comment_updated", "webhook.group.comment.updated", CommentUpdatedEvent.class))
                            .addEvent(event("comment_deleted", "webhook.group.comment.deleted", CommentDeletedEvent.class))
                            .build();
            
                    WebHookEventGroup worklogGroup = WebHookEventGroup.builder()
                            .nameI18nKey("webhook.group.worklog")
                            .addEvent(event("worklog_created", "webhook.group.worklog.created", WorklogCreatedEvent.class))
                            .addEvent(event("worklog_updated", "webhook.group.worklog.updated", WorklogUpdatedEvent.class))
                            .addEvent(event("worklog_deleted", "webhook.group.worklog.deleted", WorklogDeletedEvent.class))
                            .build();
            

            As near as I can see - there are event type ids for comments in jira-project/jira-components/jira-api/src/main/java/com/atlassian/jira/event/type/EventType.java

                public static final Long ISSUE_CREATED_ID = 1L;
                public static final Long ISSUE_UPDATED_ID = 2L;
                public static final Long ISSUE_ASSIGNED_ID = 3L;
                public static final Long ISSUE_RESOLVED_ID = 4L;
                public static final Long ISSUE_CLOSED_ID = 5L;
                public static final Long ISSUE_COMMENTED_ID = 6L;
                public static final Long ISSUE_REOPENED_ID = 7L;
                public static final Long ISSUE_DELETED_ID = 8L;
                public static final Long ISSUE_MOVED_ID = 9L;
                public static final Long ISSUE_WORKLOGGED_ID = 10L;
                public static final Long ISSUE_WORKSTARTED_ID = 11L;
                public static final Long ISSUE_WORKSTOPPED_ID = 12L;
                public static final Long ISSUE_GENERICEVENT_ID = 13L;
                public static final Long ISSUE_COMMENT_EDITED_ID = 14L;
                public static final Long ISSUE_WORKLOG_UPDATED_ID = 15L;
                public static final Long ISSUE_WORKLOG_DELETED_ID = 16L;
                public static final Long ISSUE_COMMENT_DELETED_ID = 17L;
            

            I may try to see if what looks like a trivial addition to the code would resolve this problem on a test instance
            (assuming I can get the plugin built), but someone in a position to build jira from source should be easily able to test
            out the above.

            Nathan Neulinger added a comment - I took a look at the sources and it looks like fixing this may be simple - some notes below. I may be misinterpreting here, but it looks like just getting the comment event handlers changed to the 4 argument creator functions would address the lack of filtering. jira-project/jira-components/jira-plugins/jira-webhooks-plugin/src/main/java/com/atlassian/jira/plugins/webhooks registration/RegisteredWebHookEventFactory.java: private RegisteredWebHookEvent event( final String id, final String namei18nKey, final Class <?> eventClass) { return RegisteredWebHookEvent .withId(id) .andDisplayName(namei18nKey) .firedWhen(eventClass) .isMatchedBy(EventMatchers.ALWAYS_TRUE); } private <T> RegisteredWebHookEvent event( String id, String displayName, Class <T> eventClass, EventMatcher<? super T> matcher) { return RegisteredWebHookEvent .withId(id) .andDisplayName(displayName) .firedWhen(eventClass) .isMatchedBy(matcher); } the call to set up the event handler for issue events calls event() using the 4 argument form: WebHookEventGroup issueGroup = WebHookEventGroup.builder() .nameI18nKey( "webhook.group.issue" ) .addEvent(event( "jira:issue_created" , "webhook.group.issue.created" , IssueEvent.class, Sets.newHashSet(EventType.ISSUE_CREATED_ID))) .addEvent(event( "jira:issue_updated" , "webhook.group.issue.updated" , IssueEvent.class, Sets.newHashSet(EventType.ISSUE_ASSIGNED_ID, EventType.ISSUE_CLOSED_ID, EventType.ISSUE_REOPENED_ID, EventType.ISSUE_RESOLVED_ID, EventType.ISSUE_COMMENT_EDITED_ID, EventType.ISSUE_COMMENTED_ID, EventType.ISSUE_MOVED_ID, EventType.ISSUE_UPDATED_ID, EventType.ISSUE_WORKSTARTED_ID, EventType.ISSUE_WORKSTOPPED_ID, EventType.ISSUE_GENERICEVENT_ID, EventType.ISSUE_COMMENT_DELETED_ID))) .addEvent(event( "jira:issue_deleted" , "webhook.group.issue.deleted" , IssuePreDeleteEvent.class, Collections.< Long >emptySet())) .addEvent(event( "jira:worklog_updated" , "webhook.group.issue.worklog.changed" , IssueEvent.class, Sets.newHashSet(EventType.ISSUE_WORKLOG_UPDATED_ID, EventType.ISSUE_WORKLOG_DELETED_ID, EventType.ISSUE_WORKLOGGED_ID))) .build(); where the comment and worklog handlers use the 3 argument form with the corresponding "EventMatches.ALWAYS_TRUE": WebHookEventGroup commentGroup = WebHookEventGroup.builder() .nameI18nKey( "webhook.group.comment" ) .addEvent(event( "comment_created" , "webhook.group.comment.created" , CommentCreatedEvent.class)) .addEvent(event( "comment_updated" , "webhook.group.comment.updated" , CommentUpdatedEvent.class)) .addEvent(event( "comment_deleted" , "webhook.group.comment.deleted" , CommentDeletedEvent.class)) .build(); WebHookEventGroup worklogGroup = WebHookEventGroup.builder() .nameI18nKey( "webhook.group.worklog" ) .addEvent(event( "worklog_created" , "webhook.group.worklog.created" , WorklogCreatedEvent.class)) .addEvent(event( "worklog_updated" , "webhook.group.worklog.updated" , WorklogUpdatedEvent.class)) .addEvent(event( "worklog_deleted" , "webhook.group.worklog.deleted" , WorklogDeletedEvent.class)) .build(); As near as I can see - there are event type ids for comments in jira-project/jira-components/jira-api/src/main/java/com/atlassian/jira/event/type/EventType.java public static final Long ISSUE_CREATED_ID = 1L; public static final Long ISSUE_UPDATED_ID = 2L; public static final Long ISSUE_ASSIGNED_ID = 3L; public static final Long ISSUE_RESOLVED_ID = 4L; public static final Long ISSUE_CLOSED_ID = 5L; public static final Long ISSUE_COMMENTED_ID = 6L; public static final Long ISSUE_REOPENED_ID = 7L; public static final Long ISSUE_DELETED_ID = 8L; public static final Long ISSUE_MOVED_ID = 9L; public static final Long ISSUE_WORKLOGGED_ID = 10L; public static final Long ISSUE_WORKSTARTED_ID = 11L; public static final Long ISSUE_WORKSTOPPED_ID = 12L; public static final Long ISSUE_GENERICEVENT_ID = 13L; public static final Long ISSUE_COMMENT_EDITED_ID = 14L; public static final Long ISSUE_WORKLOG_UPDATED_ID = 15L; public static final Long ISSUE_WORKLOG_DELETED_ID = 16L; public static final Long ISSUE_COMMENT_DELETED_ID = 17L; I may try to see if what looks like a trivial addition to the code would resolve this problem on a test instance (assuming I can get the plugin built), but someone in a position to build jira from source should be easily able to test out the above.

            This seems like a serious security issue. Many users expect that the JQL filter can be relied upon to prevent sensitive data from their JIRA server from being sent in webhooks. If the JQL doesn't work sometimes that opens the possibility of inadvertent information disclosure.

            Chris Waters added a comment - This seems like a serious security issue. Many users expect that the JQL filter can be relied upon to prevent sensitive data from their JIRA server from being sent in webhooks. If the JQL doesn't work sometimes that opens the possibility of inadvertent information disclosure.

            Just to add to Sami's note, we have to shut down our integration and are now contemplating a move to GitHub issues given their webhook integration is much better. I have tried for days to get https://ecosystem.atlassian.net/browse/ACJIRA-1225 rolled back for our cloud instance to no avail. This won't allow us to move forward. For our use case, the webhook integration is now completely broken.

            Andy Czerwonka added a comment - Just to add to Sami's note, we have to shut down our integration and are now contemplating a move to GitHub issues given their webhook integration is much better. I have tried for days to get  https://ecosystem.atlassian.net/browse/ACJIRA-1225  rolled back for our cloud instance to no avail. This won't allow us to move forward. For our use case, the webhook integration is now completely broken.

            This has become a really critical issue for use now because of https://ecosystem.atlassian.net/browse/ACJIRA-1225 is rolled out. As the issue events don't anymore contain comments, we need to start triggering also comment events. We have separate webhooks configured for each project with proper jql filter and now each of them is being triggered for each change, which breaks our integrations badly. 

            Sami Heikkinen added a comment - This has become a really critical issue for use now because of https://ecosystem.atlassian.net/browse/ACJIRA-1225  is rolled out. As the issue events don't anymore contain comments, we need to start triggering also comment events. We have separate webhooks configured for each project with proper jql filter and now each of them is being triggered for each change, which breaks our integrations badly. 

              istankiewicz Eve (Inactive)
              yokamoto Yuki Okamoto (Inactive)
              Affected customers:
              32 This affects my team
              Watchers:
              47 Start watching this issue

                Created:
                Updated:
                Resolved: