Add separate logging mechanism for records skipped during backup/restore

XMLWordPrintable

    • 1
    • 5

      During space/site backup/restore Confluence could skip some records for various reasons. For e.g. record could be invalid, it's dependencies couldn't be resolved or attachment file couldn't be found. This information about skipped records is added to the atlassian-confluence.log file. When process finishes users could see in UI how many records were skipped. If user wants to check what exactly was skipped and whether they are OK with the fact that records aren't included in backup or not restored, they will need to search through the whole confluence log file to be able to find what exactly was skipped.

      I suggest to consider introducing additional place where this information could be stored, so user could access it without going through all logs. For e.g. it could be:

      a) Additional file related to the job (will be deleted after 2 weeks together with the job)

      b) Backup/Restore service will create Confluence page which will be mentioned in job. When job is deleted this page could also be removed (we will need to have a special space for such pages)

       

      Example log messages:

      Class Level Category Example Log Message
      StatisticsCollector WARN Restore / Counted Restore process skipped 15 objects. To mute the full list of skipped objects, change logging to WARN for class com.atlassian.confluence.impl.backuprestore.statistics.StatisticsCollector
      StatisticsCollector INFO Restore / Counted {{Object was skipped with a reason INVALID_FIELDS and message: Value cannot be null. Object: ImportedObjectV2{id=98765, entityClass=class com.atlassian.confluence.pages.Page, originalPropertyValueMap=
      Unknown macro: {title=null, ...}

      }}}
      StatisticsCollector INFO Restore / Counted {{Object was skipped with a reason PARENT_WAS_NOT_PERSISTED: ImportedObjectV2{id=98765, entityClass=class com.atlassian.confluence.pages.Comment, originalPropertyValueMap=
      Unknown macro: {...}

      }}}
      StatisticsCollector INFO Restore / Counted {{Object was skipped with a reason NOT_SATISFIED_DEPENDENCIES: ImportedObjectV2{id=98765, entityClass=class com.atlassian.confluence.pages.Attachment, originalPropertyValueMap=
      Unknown macro: {...}

      }}}
      StatisticsCollector INFO Restore / Counted {{Object was skipped with a reason SPACE_IS_NOT_ALLOWED: ImportedObjectV2{id=98765, entityClass=class com.atlassian.confluence.spaces.Space, originalPropertyValueMap=
      Unknown macro: {...}

      }}}
      StatisticsCollector INFO Restore / Counted {{Object was skipped with a reason PARENT_NO_LONGER_EXISTS: ImportedObjectV2{id=98765, entityClass=class com.atlassian.confluence.core.ContentProperty, originalPropertyValueMap=
      Unknown macro: {...}

      }}}
      DatabasePersister WARN Restore / Counted {{Object has an empty obligatory field and will be skipped. Field: HibernateField {propertyName='title', nullable=false}

      , Object: ImportedObjectV2

      {id=98765, entityClass=class com.atlassian.confluence.pages.Page, ...}

      }}
      BodyContentPersister WARN Restore / Counted BodyContentPersister detected invalid BodyContent(s) during import which were ignored.
      BodyContentPersister DEBUG Restore / Counted Skipped BodyContent 98765 due to Failed to extract body from BodyContent
      ContentPropertiesPersister WARN Restore / Counted ContentPropertiesPersister detected invalid contentProperties during import which were ignored.
      DirectoryMappingPersister WARN Restore / Counted Couldn't resolve listIndex property value for directory mapping with id 98765
      InternalUserCredentialRecordPersister WARN Restore / Counted Couldn't resolve listIndex property value for internal user credential record with id 98765
      HibernateEntityExistenceFilter INFO Restore / Counted An object with class com.atlassian.confluence.spaces.SpaceGroup found in the input XML backup file is not known by Hibernate and will be skipped. It usually happens when the class was deprecated in previous Confluence versions. All the following objects of the same class will be skipped silently.
      ClassShouldBeExportableFilter INFO Restore / Counted An object with class com.atlassian.confluence.some.OldClass found in the input XML backup file marked as NotExportable and will be skipped. It usually happens when a class was NOT marked as NotExportable in previous Confluence versions. All the following objects of the same class will be skipped silently.
      ImportedObjectV2 WARN Restore / Counted Unexpected imported property 'legacyField' will be skipped. Entity class 'com.atlassian.confluence.pages.Page', id '98765'. (property skipped, not the whole object)
      ImportedObjectV1ToV2Converter ERROR Restore / Counted {{fromLegacyImportedObject failed, the object will be skipped: ImportedObject {class='Page', id='98765', ...}

      }}
      InvalidBandanaRecordFilter DEBUG Restore / Counted {{Imported Bandana record was skipped because either context or key field are nulls ImportedObjectV2 {id=98765, entityClass=class com.atlassian.confluence.setup.bandana.ConfluenceBandanaRecord, ...}

      }}
      XMLBackupContainerReader DEBUG Restore / Counted {{This input object has been skipped because it had been rejected by the converter. Object: ImportedObject {class='SomeClass', id='98765', ...}

      }}
      AttachmentsPersister WARN Restore / Not counted Cannot restore the attachment with id 98765. No entry in the zip file. Looked in attachments/11111/22222/1
      AttachmentsPersister WARN Restore / Not counted Cannot find in database an attachment with restored id 98765
      AttachmentsPersister WARN Restore / Not counted Cannot restore the attachment. Failed to read the stream for entry attachments/11111/22222/1.
      RestoreDao WARN Restore / Not counted Unable to persist batch of records. Records will be inserted one by one: Duplicate entry '98765' for key 'PRIMARY'
      RestoreDao WARN Restore / Not counted Record was not persisted to the database: could not execute statement; SQL [n/a]; constraint [PRIMARY]
      ImportedObjectsDispatcher WARN Restore / Not counted No persisters found for the entity with class class com.atlassian.confluence.some.LegacyEntity
      EntityInfoSqlHelper WARN Restore / Not counted Unable to parse value, null value will be used instead. Type: TimestampType, value: 2024-13-45. Exception: java.text.ParseException: Unparseable date
      EntityInfoSqlHelper WARN Restore / Not counted Unexpected type came for parsing. Null value will be used instead. Type: SomeUnknownType, value: someValue
      EntityInfoSqlHelper WARN Restore / Not counted Unexpected Component property type came for parsing. Null value will be used instead. Type: ComponentType, value: someValue
      DeferredUpdateRecordAction WARN Restore / Not counted Unable to find new id for entity class class com.atlassian.confluence.spaces.Space with original id 98765. It was not found or persisted for some reasons?
      DeferredUpdateRecordAction WARN Restore / Not counted Unable to perform update operation for entity class class com.atlassian.confluence.spaces.Space with original id 98765. The new id 11111 is found but the entity was not persisted by this moment.
      BodyContentPersister WARN Restore / Not counted Unable to find user 'abc123def456' mentioned in the body content with id '98765'.
      BodyContentPersister WARN Restore / Not counted Failed to extract user references from BodyContent with id 98765
      ClassExistenceValidator INFO Restore / Not counted Unable to instantiate 'com.atlassian.confluence.some.RemovedClass' class found in the input XML backup file. It usually happens when a class was deprecated and then removed from Confluence. All following objects of the same class will be skipped silently.
      LabelFinder WARN Restore / Not counted We found two or more similar labels. Space restore will use label with lower id (98765). Please consider removing other labels with the same name, owner and namespace
      PluginDataReader WARN Restore / Not counted No plugin data found for module descriptor: com.example.plugin:my-module
      PluginDataReader INFO Restore / Not counted Plugin data import will be skipped because the plugin com.example.plugin version 3.0.0 is required, and you are using 2.5.0. The backup was created with version 3.1.0.
      AbstractDatabaseDataConverter INFO Backup / Counted {{Object was skipped with a reason INVALID_FIELDS and message: Cannot convert value 'xyz' to Long. Object: DbRawObjectData {table='CONTENT', id=98765, ...}

      }}
      XmlBackupContainerWriter WARN Backup / Not counted Attachment with id 98765 not found in database.
      XmlBackupContainerWriter WARN Backup / Not counted Attachment file not found. Attachment data not found for attachment ref [id=98765, ...]
      XmlBackupContainerWriter ERROR Backup / Not counted {{Unexpected exception while backup up an attachment AttachmentInfo {id=98765, containerId=11111, version=1}

      . The file might be skipped from the backup.}}

              Assignee:
              Unassigned
              Reporter:
              Irina Tiapchenko (Inactive)
              Votes:
              8 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: