Uploaded image for project: 'Jira Service Management Data Center'
  1. Jira Service Management Data Center
  2. JSDSERVER-12010

After running some Insight imports, some objects have duplicated or more attribute values on object attribute that should only allow exactly 1 value (maximum cardinality = 1)

      Issue Summary

      After running some Insight imports, some objects have duplicated or more attribute values on object attribute that should only allow exactly 1 value (maximum cardinality = 1)

      This is reproducible on Data Center: yes

      Steps to Reproduce

      N/A. This issue couldn't be reproduced on the UI, it usually happens after some Insight imports

      Expected Results

      All attributes that have maximum cardinality = 1 have only 1 value

      Actual Results

      There are 2 or more values on the attributes that have maximum cardinality = 1

      Workaround

      1. Follow the steps in Duplicated attribute values in Insight Objects with cardinality maximum 1
      2. Run the SQL queries below to identify duplicated attribute and attribute values. If the SELECT statement returns something, run the DELETE statement. After that, run the same SELECT statement again (and DELETE if anything returns again) until it returns nothing.
        select * from "AO_8542F1_IFJ_OBJ_ATTR_VAL" where "OBJECT_ATTRIBUTE_ID" in (
        SELECT min(OA2."ID") AS TO_DELETE
        FROM "AO_8542F1_IFJ_OBJ_ATTR" OA2
                 RIGHT JOIN (
            SELECT OTA."ID" as OTA_ID, OTA."NAME", OA."OBJECT_ID" as OA_OID, count(*)
            FROM "AO_8542F1_IFJ_OBJ_TYPE_ATTR" OTA
                     LEFT JOIN "AO_8542F1_IFJ_OBJ_ATTR" OA ON OTA."ID" = OA."OBJECT_TYPE_ATTRIBUTE_ID"
            GROUP BY OTA."ID", OTA."NAME", OA."OBJECT_ID"
            HAVING count(*) > 1) AS SUBQ ON OA2."OBJECT_TYPE_ATTRIBUTE_ID" = SUBQ.OTA_ID AND OA2."OBJECT_ID" = SUBQ.OA_OID
        GROUP BY OA2."OBJECT_ID", OA2."OBJECT_TYPE_ATTRIBUTE_ID");
        
        DELETE from "AO_8542F1_IFJ_OBJ_ATTR_VAL" where "OBJECT_ATTRIBUTE_ID" in (
        SELECT min(OA2."ID") AS TO_DELETE
        FROM "AO_8542F1_IFJ_OBJ_ATTR" OA2
                 RIGHT JOIN (
            SELECT OTA."ID" as OTA_ID, OTA."NAME", OA."OBJECT_ID" as OA_OID, count(*)
            FROM "AO_8542F1_IFJ_OBJ_TYPE_ATTR" OTA
                     LEFT JOIN "AO_8542F1_IFJ_OBJ_ATTR" OA ON OTA."ID" = OA."OBJECT_TYPE_ATTRIBUTE_ID"
            GROUP BY OTA."ID", OTA."NAME", OA."OBJECT_ID"
            HAVING count(*) > 1) AS SUBQ ON OA2."OBJECT_TYPE_ATTRIBUTE_ID" = SUBQ.OTA_ID AND OA2."OBJECT_ID" = SUBQ.OA_OID
        GROUP BY OA2."OBJECT_ID", OA2."OBJECT_TYPE_ATTRIBUTE_ID");
        
        select * FROM "AO_8542F1_IFJ_OBJ_ATTR"
        where "ID" in (
        SELECT min(OA2."ID") AS TO_DELETE
        FROM "AO_8542F1_IFJ_OBJ_ATTR" OA2
                 RIGHT JOIN (
            SELECT OTA."ID" as OTA_ID, OTA."NAME", OA."OBJECT_ID" as OA_OID, count(*)
            FROM "AO_8542F1_IFJ_OBJ_TYPE_ATTR" OTA
                     LEFT JOIN "AO_8542F1_IFJ_OBJ_ATTR" OA ON OTA."ID" = OA."OBJECT_TYPE_ATTRIBUTE_ID"
            GROUP BY OTA."ID", OTA."NAME", OA."OBJECT_ID"
            HAVING count(*) > 1) AS SUBQ ON OA2."OBJECT_TYPE_ATTRIBUTE_ID" = SUBQ.OTA_ID AND OA2."OBJECT_ID" = SUBQ.OA_OID
        GROUP BY OA2."OBJECT_ID", OA2."OBJECT_TYPE_ATTRIBUTE_ID");
        
        DELETE FROM "AO_8542F1_IFJ_OBJ_ATTR"
        where "ID" in (
        SELECT min(OA2."ID") AS TO_DELETE
        FROM "AO_8542F1_IFJ_OBJ_ATTR" OA2
                 RIGHT JOIN (
            SELECT OTA."ID" as OTA_ID, OTA."NAME", OA."OBJECT_ID" as OA_OID, count(*)
            FROM "AO_8542F1_IFJ_OBJ_TYPE_ATTR" OTA
                     LEFT JOIN "AO_8542F1_IFJ_OBJ_ATTR" OA ON OTA."ID" = OA."OBJECT_TYPE_ATTRIBUTE_ID"
            GROUP BY OTA."ID", OTA."NAME", OA."OBJECT_ID"
            HAVING count(*) > 1) AS SUBQ ON OA2."OBJECT_TYPE_ATTRIBUTE_ID" = SUBQ.OTA_ID AND OA2."OBJECT_ID" = SUBQ.OA_OID
        GROUP BY OA2."OBJECT_ID", OA2."OBJECT_TYPE_ATTRIBUTE_ID");
        
      3. When both SELECT statements above return nothing anymore, run an Insight reindex to keep the index updated.

          Form Name

            [JSDSERVER-12010] After running some Insight imports, some objects have duplicated or more attribute values on object attribute that should only allow exactly 1 value (maximum cardinality = 1)

            Same issue running 9.12.4. Ran the health checker and it found over 6000 duplicates.

            Never knew this health checker existed until now. Surely should be available on system admin menus.

            James Beagrie added a comment - Same issue running 9.12.4. Ran the health checker and it found over 6000 duplicates. Never knew this health checker existed until now. Surely should be available on system admin menus.

            Gonchik Tsymzhitov added a comment - - edited

            It will be nice to see that ticket as fixed.
            5.12.5 is waiting issue

            Gonchik Tsymzhitov added a comment - - edited It will be nice to see that ticket as fixed. 5.12.5 is waiting issue

            Version 5.12.2 is impacted as well.

            Justin Deutsch added a comment - Version 5.12.2 is impacted as well.

            We discovered a number of duplicated values on our system. Among the attributes with duplicated value is the "Updated" attribute, which is maintained by Assets automatically.

            Since we are running a cluster with multiple nodes and have seen other issues with the index synchronization, I suspect the node synchronization to contribute to this problem.

            Martin Bickel added a comment - We discovered a number of duplicated values on our system. Among the attributes with duplicated value is the "Updated" attribute, which is maintained by Assets automatically. Since we are running a cluster with multiple nodes and have seen other issues with the index synchronization, I suspect the node synchronization to contribute to this problem.

            We have this on 5.4.12 on DC

            Gonchik Tsymzhitov added a comment - We have this on 5.4.12 on DC

            Antreas Solou added a comment - - edited

            We have this as well. Version 5.4.11 on Data Center. 

            Antreas Solou added a comment - - edited We have this as well. Version 5.4.11 on Data Center. 

            Hi All,

            I just realised I didn't mention we are using JSM 5.4.11, for context.

            Thanks

            Justin Deutsch added a comment - Hi All, I just realised I didn't mention we are using JSM 5.4.11, for context. Thanks

            Hi All,

            This was impacting one of our clients in only production and we needed to be able to replicate it in lower environments to be able identify fixes.

            I'm fairly confident I have been able to replicate this using Groovy Runner scripts to update a single object. The logic of the script is (and is run from two browsers simultaneously):

            1. get a list of all the valid objects for the attribute
            2. for 10,000 iterations on ObjectA, do:
              1. select an object to set (ObjectB)
              2. set the attribute on ObejctA to ObjectB

            Once I have run the scripts, I'm seeing multiple different objects in the attribute which should only contain at most one object. I believe the reason there is a difference is because of random selection of a valid object to set the attribute value to, where as an import is likely to be attempting to set the same values if two imports are run at the same time.

            The dark feature in Duplicated attribute values in Insight Objects with cardinality maximum 1 doesn't detect the extra objects, most likely because the script updates ObjectA 10,000 times each from two browsers and by the time the script finishes the extra objects have been removed. I catch the “duplication” by refreshing the object view screen, while the scripts are running.

            The SQL provided in Duplicated attribute values in Insight Objects with cardinality maximum 1 does detect that there were extra values and which can be cleaned up.

            jiradb=# SELECT "OBJECT_ID", count("OBJECT_ID") FROM "AO_8542F1_IFJ_OBJ_ATTR" WHERE "OBJECT_TYPE_ATTRIBUTE_ID"=980 GROUP BY "OBJECT_ID" HAVING count("OBJECT_ID") > 1;
             OBJECT_ID | count
            -----------+-------
                  8206 | 7
                  8204 | 2 
            (2 rows)

            We have also seen attributes which can hold multiple objects have object duplication.

            I hope this helps others.

            Justin Deutsch added a comment - Hi All, This was impacting one of our clients in only production and we needed to be able to replicate it in lower environments to be able identify fixes. I'm fairly confident I have been able to replicate this using Groovy Runner scripts to update a single object. The logic of the script is (and is run from two browsers simultaneously): get a list of all the valid objects for the attribute for 10,000 iterations on ObjectA, do: select an object to set (ObjectB) set the attribute on ObejctA to ObjectB Once I have run the scripts, I'm seeing multiple different objects in the attribute which should only contain at most one object. I believe the reason there is a difference is because of random selection of a valid object to set the attribute value to, where as an import is likely to be attempting to set the same values if two imports are run at the same time. The dark feature in Duplicated attribute values in Insight Objects with cardinality maximum 1 doesn't detect the extra objects, most likely because the script updates ObjectA 10,000 times each from two browsers and by the time the script finishes the extra objects have been removed. I catch the “duplication” by refreshing the object view screen, while the scripts are running. The SQL provided in Duplicated attribute values in Insight Objects with cardinality maximum 1 does detect that there were extra values and which can be cleaned up. jiradb=# SELECT "OBJECT_ID" , count( "OBJECT_ID" ) FROM "AO_8542F1_IFJ_OBJ_ATTR" WHERE "OBJECT_TYPE_ATTRIBUTE_ID" =980 GROUP BY "OBJECT_ID" HAVING count( "OBJECT_ID" ) > 1; OBJECT_ID | count -----------+------- 8206 | 7 8204 | 2 (2 rows) We have also seen attributes which can hold multiple objects have object duplication. I hope this helps others.

            It would be good to know why this occurs so we can look at how we can avoid trigging the bug.

            Justin Deutsch added a comment - It would be good to know why this occurs so we can look at how we can avoid trigging the bug.

            Does anyone know what the cause of this issue is or if there's going to be a better fix? We have an issue where the "updated" attribute is duplicated, even though it should be cardinality=1 (as a system attribute). I'm not very keen on the idea of running queries on the database to fix the issue.

            Amos Bennett added a comment - Does anyone know what the cause of this issue is or if there's going to be a better fix? We have an issue where the "updated" attribute is duplicated, even though it should be cardinality=1 (as a system attribute). I'm not very keen on the idea of running queries on the database to fix the issue.

              aba1389a0947 Josh Cameron
              michin Michelle Chin
              Affected customers:
              39 This affects my team
              Watchers:
              42 Start watching this issue

                Created:
                Updated:
                Resolved: