Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-95859

Confluence Webhook Admin UI Raises an Error when a Webhook event happens

XMLWordPrintable

      Problem

      Confluence Webhook Admin UI Raises an Error when a Webhook event happens

      Environment

      Confluence 8.8.1 & 8.9.1 - Any OS

      Steps to Reproduce

      Create a webhook that uses attachment_created or attachment_updated event. It may happen with other events, I didn't test all.

      Expected Results

      Confluence Webook Admin UI should show the historical webook usage

      Actual Results

      Confluence Webook Admin UI raises an error.

      Workaround

      Restart Confluence instance allows the use of webhooks admin Ui until a webhook is triggered. However, the historical information about the executed webhooks will be inaccurate.

      Notes

      Versions 7.19.22, 8.5.9 and 8.7.2 aren't affected by this bug. On those versions, with net.java.ao DEBUG enabled, we can see the following messages on logs

      2024-05-20 15:48:42,517 INFO [http-nio-21922-exec-4 url: /c71922/pages/doattachfile.action; user: admin] [synchrony.service.http.SynchronyRequestExecutor] execute Initiating request PUT http://127.0.0.1:41922/synchrony/v1/data/Synchrony-0060a06a-31c6-3b41-9013-fd3d70bb9eb6/confluence-98319?state-at=@head&state-format=html HTTP/1.1 for 98319
2024-05-20 15:48:42,698 DEBUG [http-nio-21922-exec-6 url: /c71922/pages/viewpageattachments.action; user: admin] [net.java.ao.sql] onSql SELECT "NAME","NODE_ID","ID","PORT_NUMBER","IP_ADDRESS" FROM "AO_59F889_ZDU_CLUSTER_NODES"
2024-05-20 15:48:42,700 DEBUG [http-nio-21922-exec-6 url: /c71922/pages/viewpageattachments.action; user: admin] [net.java.ao.sql] onSql SELECT "DATE","ID","USER_KEY","PLUGIN_KEY","KEY" FROM "AO_6384AB_DISCOVERED" WHERE "USER_KEY" = ?
2024-05-20 15:48:42,708 DEBUG [http-nio-21922-exec-6 url: /c71922/pages/viewpageattachments.action; user: admin] [net.java.ao.sql] onSql SELECT "NAME","NODE_ID","ID","PORT_NUMBER","IP_ADDRESS" FROM "AO_59F889_ZDU_CLUSTER_NODES"
2024-05-20 15:48:42,710 DEBUG [http-nio-21922-exec-6 url: /c71922/pages/viewpageattachments.action; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:42,711 DEBUG [http-nio-21922-exec-6 url: /c71922/pages/viewpageattachments.action; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:42,724 DEBUG [http-nio-21922-exec-6 url: /c71922/pages/viewpageattachments.action; user: admin] [net.java.ao.sql] onSql SELECT message."ID",message."CONTENT" FROM "AO_21F425_MESSAGE_AO" message JOIN "AO_21F425_MESSAGE_MAPPING_AO" message_mapping ON message."ID" = message_mapping."MESSAGE_ID" WHERE "USER_HASH" = ? OR "USER_HASH" = ? OR "USER_HASH" = ? OR "USER_HASH" = ?
2024-05-20 15:48:42,727 DEBUG [http-nio-21922-exec-6 url: /c71922/pages/viewpageattachments.action; user: admin] [net.java.ao.sql] onSql SELECT message."ID",message."CONTENT" FROM "AO_21F425_MESSAGE_AO" message JOIN "AO_21F425_MESSAGE_MAPPING_AO" message_mapping ON message."ID" = message_mapping."MESSAGE_ID" WHERE "USER_HASH" = ? OR "USER_HASH" = ? OR "USER_HASH" = ? OR "USER_HASH" = ?
2024-05-20 15:48:42,928 DEBUG [http-nio-21922-exec-5 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "NAME","NODE_ID","ID","PORT_NUMBER","IP_ADDRESS" FROM "AO_59F889_ZDU_CLUSTER_NODES"
2024-05-20 15:48:42,931 DEBUG [http-nio-21922-exec-5 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:42,932 DEBUG [http-nio-21922-exec-5 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:42,961 DEBUG [http-nio-21922-exec-1 url: /c71922/rest/mywork/latest/status/notification/count; user: admin] [net.java.ao.sql] onSql SELECT "ACTION","UPDATED","GLOBAL_ID","METADATA","ACTION_ICON_URL","ITEM_ICON_URL","DESCRIPTION","GROUPING_ID","READ","ENTITY","ICON_URL","PINNED","TITLE","APPLICATION_LINK_ID","ITEM_TITLE","URL","ID","CREATED","APPLICATION","ITEM_URL","USER","STATUS" FROM "AO_9412A1_AONOTIFICATION" WHERE "USER" = ? AND "APPLICATION_LINK_ID" = ? AND "APPLICATION" = ? AND "READ" <> ?
2024-05-20 15:48:42,973 DEBUG [http-nio-21922-exec-7 url: /c71922/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT "ID","USER_KEY" FROM "AO_4789DD_HEALTH_CHECK_WATCHER" WHERE "USER_KEY" = ?
2024-05-20 15:48:42,975 DEBUG [http-nio-21922-exec-7 url: /c71922/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT "IS_HEALTHY","APPLICATION_NAME","STATUS_NAME","FAILED_DATE","SEVERITY","COMPLETE_KEY","ID","IS_RESOLVED","RESOLVED_DATE","DESCRIPTION","FAILURE_REASON","NODE_ID" FROM "AO_4789DD_HEALTH_CHECK_STATUS" WHERE "IS_HEALTHY" = ? AND "IS_RESOLVED" = ? ORDER BY "SEVERITY" ASC
2024-05-20 15:48:42,976 DEBUG [http-nio-21922-exec-7 url: /c71922/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT "SNOOZE_DATE","NOTIFICATION_ID","ID","USER_KEY","SNOOZE_COUNT","IS_SNOOZED" FROM "AO_4789DD_READ_NOTIFICATIONS" WHERE "USER_KEY" = ? AND "IS_SNOOZED" = ?
2024-05-20 15:48:42,979 DEBUG [http-nio-21922-exec-7 url: /c71922/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT "PROPERTY_VALUE","PROPERTY_NAME","ID" FROM "AO_4789DD_PROPERTIES" WHERE "PROPERTY_NAME" = ?
2024-05-20 15:48:42,980 DEBUG [http-nio-21922-exec-7 url: /c71922/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT "SNOOZE_DATE","NOTIFICATION_ID","ID","USER_KEY","SNOOZE_COUNT","IS_SNOOZED" FROM "AO_4789DD_READ_NOTIFICATIONS" WHERE "USER_KEY" = ? AND "IS_SNOOZED" = ? AND "SNOOZE_DATE" > ?
2024-05-20 15:48:43,067 DEBUG [http-nio-21922-exec-4 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "NAME","NODE_ID","ID","PORT_NUMBER","IP_ADDRESS" FROM "AO_59F889_ZDU_CLUSTER_NODES"
2024-05-20 15:48:43,071 DEBUG [http-nio-21922-exec-4 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:43,073 DEBUG [http-nio-21922-exec-4 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:43,151 DEBUG [http-nio-21922-exec-10 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "NAME","NODE_ID","ID","PORT_NUMBER","IP_ADDRESS" FROM "AO_59F889_ZDU_CLUSTER_NODES"
2024-05-20 15:48:43,154 DEBUG [http-nio-21922-exec-10 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:43,155 DEBUG [http-nio-21922-exec-10 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:43,286 DEBUG [http-nio-21922-exec-5 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "NAME","NODE_ID","ID","PORT_NUMBER","IP_ADDRESS" FROM "AO_59F889_ZDU_CLUSTER_NODES"
2024-05-20 15:48:43,288 DEBUG [http-nio-21922-exec-5 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:48:43,290 DEBUG [http-nio-21922-exec-5 url: /c71922/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT "ID","CONTENT" FROM "AO_21F425_MESSAGE_AO" LIMIT 1
2024-05-20 15:49:09,190 DEBUG [atlassian-webhooks-scheduler:thread-1] [net.java.ao.sql] onSql SELECT "WEBHOOK_ID","ID","FINISH" FROM "AO_A0B856_HIST_INVOCATION" WHERE "ID" = ?
2024-05-20 15:49:09,195 DEBUG [atlassian-webhooks-scheduler:thread-1] [net.java.ao.sql] onSql DELETE FROM "AO_A0B856_HIST_INVOCATION" WHERE "ID" IN (?)
2024-05-20 15:49:09,197 DEBUG [atlassian-webhooks-scheduler:thread-1] [net.java.ao.sql] onSql INSERT INTO "AO_A0B856_HIST_INVOCATION" ("OUTCOME","REQUEST_URL","WEBHOOK_ID","RESULT_DESCRIPTION","RESPONSE_HEADERS","START","EVENT_ID","FINISH","REQUEST_BODY","REQUEST_ID","ID","REQUEST_HEADERS","REQUEST_METHOD","STATUS_CODE") VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)
2024-05-20 15:49:09,199 DEBUG [atlassian-webhooks-scheduler:thread-1] [net.java.ao.sql] onSql SELECT "WEBHOOK_ID","DAY_SINCE_EPOCH","EVENT_ID","ERRORS","ID","FAILURES","SUCCESSES" FROM "AO_A0B856_DAILY_COUNTS" WHERE "ID" = ?
2024-05-20 15:49:09,201 DEBUG [atlassian-webhooks-scheduler:thread-1] [net.java.ao.sql] onSql DELETE FROM "AO_A0B856_DAILY_COUNTS" WHERE "ID" IN (?)
2024-05-20 15:49:09,202 DEBUG [atlassian-webhooks-scheduler:thread-1] [net.java.ao.sql] onSql INSERT INTO "AO_A0B856_DAILY_COUNTS" ("ERRORS","FAILURES","WEBHOOK_ID","ID","DAY_SINCE_EPOCH","SUCCESSES","EVENT_ID") VALUES (?,?,?,?,?,?,?)

      However, on Confluence 8.8.1 and 8.9.1, the error happens.

      2024-05-20 14:54:53,591 DEBUG [http-nio-6881-exec-8 url: /c881/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT \"SNOOZE_DATE\",\"NOTIFICATION_ID\",\"ID\",\"USER_KEY\",\"SNOOZE_COUNT\",\"IS_SNOOZED\" FROM \"AO_4789DD_READ_NOTIFICATIONS\" WHERE \"USER_KEY\" = ? AND \"IS_SNOOZED\" = ?
2024-05-20 14:54:53,591 DEBUG [http-nio-6881-exec-8 url: /c881/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT \"PROPERTY_VALUE\",\"PROPERTY_NAME\",\"ID\" FROM \"AO_4789DD_PROPERTIES\" WHERE \"PROPERTY_NAME\" = ?
2024-05-20 14:54:53,592 DEBUG [http-nio-6881-exec-8 url: /c881/rest/troubleshooting/1.0/check/admin; user: admin] [net.java.ao.sql] onSql SELECT \"SNOOZE_DATE\",\"NOTIFICATION_ID\",\"ID\",\"USER_KEY\",\"SNOOZE_COUNT\",\"IS_SNOOZED\" FROM \"AO_4789DD_READ_NOTIFICATIONS\" WHERE \"USER_KEY\" = ? AND \"IS_SNOOZED\" = ? AND \"SNOOZE_DATE\" > ?
2024-05-20 14:54:53,907 DEBUG [http-nio-6881-exec-2 url: /c881/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT \"NAME\",\"NODE_ID\",\"ID\",\"PORT_NUMBER\",\"IP_ADDRESS\" FROM \"AO_59F889_ZDU_CLUSTER_NODES\"
2024-05-20 14:54:53,909 DEBUG [http-nio-6881-exec-2 url: /c881/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT \"ID\",\"CONTENT\" FROM \"AO_21F425_MESSAGE_AO\" LIMIT 1
2024-05-20 14:54:53,910 DEBUG [http-nio-6881-exec-2 url: /c881/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT \"ID\",\"CONTENT\" FROM \"AO_21F425_MESSAGE_AO\" LIMIT 1
2024-05-20 14:54:54,112 DEBUG [http-nio-6881-exec-6 url: /c881/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT \"NAME\",\"NODE_ID\",\"ID\",\"PORT_NUMBER\",\"IP_ADDRESS\" FROM \"AO_59F889_ZDU_CLUSTER_NODES\"
2024-05-20 14:54:54,115 DEBUG [http-nio-6881-exec-6 url: /c881/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT \"ID\",\"CONTENT\" FROM \"AO_21F425_MESSAGE_AO\" LIMIT 1
2024-05-20 14:54:54,117 DEBUG [http-nio-6881-exec-6 url: /c881/rest/wrm/2.0/resources; user: admin] [net.java.ao.sql] onSql SELECT \"ID\",\"CONTENT\" FROM \"AO_21F425_MESSAGE_AO\" LIMIT 1
2024-05-20 14:54:58,943 DEBUG [atlassian-webhooks-scheduler:thread-1] [net.java.ao.sql] onSql SELECT \"WEBHOOK_ID\",\"ID\",\"FINISH\" FROM \"AO_A0B856_HIST_INVOCATION\" WHERE \"ID\" = ?
2024-05-20 14:54:58,965 WARN [atlassian-webhooks-scheduler:thread-1] [confluence.impl.hibernate.ConfluenceHibernateTransactionManager] doRollback Performing rollback. Transactions:\n  ->[PluginReadWriteTx]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT (Session #37023577)
2024-05-20 14:54:58,965 WARN [atlassian-webhooks-scheduler:thread-1] [webhooks.internal.dao.AsyncInvocationHistoryDao] flush Failed to write webhook invocation data for 1:attachment_created to the database
java.lang.IllegalArgumentException: The follow required fields were not set when trying to create entity 'com.atlassian.webhooks.internal.dao.ao.AoHistoricalInvocation', those fields are: ImmutableFieldInfo{fieldName='EVENT_SCOPE_ID', polymorphicName='null', accessor=public abstract java.lang.String com.atlassian.webhooks.internal.dao.ao.AoHistoricalInvocation.getEventScopeId(), mutator=null, primary=false, nullable=false, autoIncrement=false, defaultValue=false, fieldType=class java.lang.String, typeInfo=String:VARCHAR, generatorType=null}, ImmutableFieldInfo{fieldName='EVENT_SCOPE_TYPE', polymorphicName='null', accessor=public abstract java.lang.String com.atlassian.webhooks.internal.dao.ao.AoHistoricalInvocation.getEventScopeType(), mutator=null, primary=false, nullable=false, autoIncrement=false, defaultValue=false, fieldType=class java.lang.String, typeInfo=String:VARCHAR, generatorType=null}

            [CONFSERVER-95859] Confluence Webhook Admin UI Raises an Error when a Webhook event happens

            Akshay Rai added a comment -

            A fix for this issue is available in Confluence Server and Data Center 8.9.5.
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            Akshay Rai added a comment - A fix for this issue is available in Confluence Server and Data Center 8.9.5. Upgrade now or check out the Release Notes to see what other issues are resolved.

            James Whitehead added a comment -

            A fix for this issue is available in Confluence Data Center 9.0.1.
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            James Whitehead added a comment - A fix for this issue is available in Confluence Data Center 9.0.1. Upgrade now or check out the Release Notes to see what other issues are resolved.

            Gary Gindlin added a comment -

            I am haing the same issue. Anyway to turn off logging of warnings to keep the logs from filling up?

            What process do I need to turn off logging for temporarily stopping these warnings.

            Gary Gindlin added a comment - I am haing the same issue. Anyway to turn off logging of warnings to keep the logs from filling up? What process do I need to turn off logging for temporarily stopping these warnings.

            Ture Hoefner {Appfire} added a comment -

            One of my 8.9.3 test server logs is so full of the error from the webhooks.internal.dao.AsyncInvocationHistoryDao that it is difficult to wade through it to find the other log lines I am looking for.  Massive logging of this:

            java.lang.IllegalArgumentException: The follow required fields were not set when trying to create entity 'com.atlassian.webhooks.internal.dao.ao.AoHistoricalInvocation

            Ture Hoefner {Appfire} added a comment - One of my 8.9.3 test server logs is so full of the error from the webhooks.internal.dao.AsyncInvocationHistoryDao that it is difficult to wade through it to find the other log lines I am looking for.  Massive logging of this: java.lang.IllegalArgumentException: The follow required fields were not set when trying to create entity 'com.atlassian.webhooks.internal.dao.ao.AoHistoricalInvocation

              79bc1a6e7b3e Archit Goyal
              8ccca53078e5 Marcelo da Costa
              Affected customers:
              8 This affects my team
              Watchers:
              11 Start watching this issue

                Created:
                Updated:
                Resolved: