Issue Summary

      Currently - missing objects are flagged per import configuration.

      So, if the same object type is being updated by two different Import configuration - both with "Missing Objects" set to update a Status Attribute, with any threshold. After the threshold is reached on either import - the Status Attribute across all objects will fluctuate, depending on which Import was running last.

      Steps to Reproduce

      1. Configure 2 Import configurations (A, B) both updating the same Object Type for example: Importing from 2 Azure Subscriptions into the same Structure, with the Predefined mapping
      2. Let each import run on schedule to accede the Threshold
      3. Missing Objects Update on some object types is set to Update the Status Attribute to "Not found", Assume threshold is 2 syncs - for the explanation below
      4. Notice Object A (included in Import A) and Object B (included in Import B) both belong to the same object type X

      Expected Results

      Requested:

      If an objects in the same object type is updated by any import, the Missing Object's Counter is reset:

      Following the Imports along a timeline:
      Import A is running creating Object A

      Import B is running creating Object B, flagging Object A as a Missing Object

      Import A is running - Identifying Object A, Resetting Object A Flag, flagging Object B as a Missing Object

      Import B is running - identifying Object B, Resetting Object B Flag, flagging Object A as a Missing Object

      and so on - the end result: Status is not updated, unless an object is missing in BOTH the Imports for the given threshold

      Actual Results

      Following the Imports along a timeline:
      Import A is running creating Object A

      Import B is running creating Object B, flagging Object A as a Missing Object

      Import A is running - Identifying Object A, flagging Object B as a Missing Object

      Import B is running - identifying Object B, updating Status of Object A to "Not Found"(Threshold was reached)

      Import A is running - identifying Object A, Updating Status of Object A to "Running", updating Status of Object B to "Not Found" (Threshold was reached)

      Import B is running - identifying Object B, Updating Status of Object B to "Running", updating Status of Object A to "Not Found" (Threshold was reached)

      and so on.

      ...
      

      Workaround

      Either create a Root Object Type per subscription, so that each Import configuration will actually update its own Object Types,(Pros - Missing Objects will act as expected, Cons - hard to maintain, troubleshoot and requires more administration)

      or;

      Set Missing Objects to Ignore and configure an Automation rule to run on Schedule, and update Status to not found, if the object was not updated for a set period (Updated < now(-2d) - assuming that each Import will actually update the Objects (Pros - disregard the Missing Objects causing the Status to fluctuate. Cons - requires and Object update as a condition, and may add to performance when running)

          Form Name

            [JSDSERVER-7225] Insight Missing Objects across all import configurations

            Craig Shannon added a comment - - edited
            Atlassian Update – 19 January 2023

            Hi everyone,

            Thank you for your feedback, we’re excited to let you know that we’re starting work on this bug on Assets import configuration.

            This improvement will ensure that objects are only flagged as missing from an import when the object was created or updated via the import being executed. This will help avoid clashes between multiple import configurations for the same object type.

            An example scenario would be:

            • CSV Import users from system A
            • CSV Import users from system B
            • JSON Import users from system C
            • Assets user manually creates / updates users

            When an import from system A runs, it should never mark users which were imported or updated from system B or C as being missing. It should also not mark the manually created users as being missing.

            The current behaviour does not consider where the object came from when marking them as missed.

            Please continue to follow this ticket for further updates as we progress this improvement.

            Kind regards,
            Craig Shannon
            Jira Service Management Data Center

            Craig Shannon added a comment - - edited Atlassian Update – 19 January 2023 Hi everyone, Thank you for your feedback, we’re excited to let you know that we’re starting work on this bug on Assets import configuration. This improvement will ensure that objects are only flagged as missing from an import when the object was created or updated via the import being executed. This will help avoid clashes between multiple import configurations for the same object type. An example scenario would be: CSV Import users from system A CSV Import users from system B JSON Import users from system C Assets user manually creates / updates users When an import from system A runs, it should never mark users which were imported or updated from system B or C as being missing. It should also not mark the manually created users as being missing. The current behaviour does not consider where the object came from when marking them as missed. Please continue to follow this ticket for further updates as we progress this improvement. Kind regards, Craig Shannon Jira Service Management Data Center

            Hello,

            it seems that there is nothing going on on this Blocker Issue?

            I don't know if it is missing on ideas to solve that situation?
            But for example we have multiple imports which affects one object type
            and also manually added objects to the same OT (not described in the ticket yet)

            Possible solution thoughts:

            If there would be an table that stores the information when and which import has "touched" an object.
            Then it would be possible to add an "Not Found" Option of:

            • Apply to any objects (which is now the case, no information store needed)
            • Apply to any imported objects (which will just affect objects that are imported by any kind of import configs, but not manually added objects)
            • Apply just to object of this import configuration (which will just handle objects that having an "internal reference" to this import config)

            I just can say please please please start handling this issue as it is so important and so critical for an CMDB
            as the data consistency is one of the highest goals in an CMDB!

            And please give us some feedback! I don't like to move to another working CMDB Solution as Insight could be the best on the marked if some critical issues would be solved (in an realistic time!!! Not decades!)

            Christian Solle added a comment - Hello, it seems that there is nothing going on on this Blocker Issue? I don't know if it is missing on ideas to solve that situation? But for example we have multiple imports which affects one object type and also manually added objects to the same OT (not described in the ticket yet) Possible solution thoughts: If there would be an table that stores the information when and which import has "touched" an object. Then it would be possible to add an "Not Found" Option of: Apply to any objects (which is now the case, no information store needed) Apply to any imported objects (which will just affect objects that are imported by any kind of import configs, but not manually added objects) Apply just to object of this import configuration (which will just handle objects that having an "internal reference" to this import config) I just can say please please please start handling this issue as it is so important and so critical for an CMDB as the data consistency is one of the highest goals in an CMDB! And please give us some feedback! I don't like to move to another working CMDB Solution as Insight could be the best on the marked if some critical issues would be solved (in an realistic time!!! Not decades!)

            Hi,

            will be there any customer communication about that core issue?

            Maybe a small information like: We understand that this is a core issue which leads into data inconsistency,
            we are working on it..... anything that I can use in communication with my managers (I like to avoid to move over to an BMC Product or Service Now )

            Christian Solle added a comment - Hi, will be there any customer communication about that core issue? Maybe a small information like: We understand that this is a core issue which leads into data inconsistency, we are working on it..... anything that I can use in communication with my managers (I like to avoid to move over to an BMC Product or Service Now )

            Hi,

            here me again.

            1) can we expect an time plan when this will be fixed or started to work on?

            maybe a quarter of a year (<10 Atlassian years 🤞)

             

            2) can we change this from a suggestion to a bug?

            This behavior is causing data inconsistency at it is a general function (promoted) to import data from several sources.

             

            // Christian  

            Christian Solle added a comment - Hi, here me again. 1) can we expect an time plan when this will be fixed or started to work on? maybe a quarter of a year (<10 Atlassian years 🤞)   2) can we change this from a suggestion to a bug? This behavior is causing data inconsistency at it is a general function (promoted) to import data from several sources.   // Christian  

            Hi,

            I just want to figure out if there is a time frame that we can plan with.

            As the actual state makes it impossible to have a consistent CMDB

            We plan to go live with the CMDB in around 6 month and this issue is a major blocker.

            Thanks for any feedback.

             

            // Christian

            Christian Solle added a comment - Hi, I just want to figure out if there is a time frame that we can plan with. As the actual state makes it impossible to have a consistent CMDB We plan to go live with the CMDB in around 6 month and this issue is a major blocker. Thanks for any feedback.   // Christian

            Yinon Negev added a comment - - edited

            A few Use Cases:
            1. AWS imports with Multiple Accounts

            2. Azure Imports with multiple subscriptions

            3. Combined Imports: Azure Integration + Discovery - both importing into the same Object Types... (Thanks CS )

            Yinon Negev added a comment - - edited A few Use Cases: 1. AWS imports with Multiple Accounts 2. Azure Imports with multiple subscriptions 3. Combined Imports: Azure Integration + Discovery - both importing into the same Object Types... (Thanks CS )

              Unassigned Unassigned
              8cdc82c96fd5 Yinon Negev
              Affected customers:
              18 This affects my team
              Watchers:
              17 Start watching this issue

                Created:
                Updated:
                Resolved: