[JSDSERVER-15364] Assets Import may not update/record progress results (AO_8542F1_IFJ_PRG_RES/AO_8542F1_IFJ_PRG_OT_RES) on MS SQLServer Database - Create and track feature requests for Atlassian products.

Type: Bug
Resolution: Fixed
Priority: Medium
Fix Version/s: 10.1.1
Affects Version/s: 5.4.7
Component/s: Assets - Import
Labels:
None

Support reference count:
3
Symptom Severity:
Severity 2 - Major
UIS:
40
Bug Fix Policy:
View Atlassian Server bug fix policy

Issue Summary

In rare circumstances, Assets imports might be running fine on schedule, but it won't be updating the import summary (process results), and the STARTED dateTime on DB table AO_8542F1_IFJ_PRG_RES is sticking to the same value.

This obscures Assets admin's visibility if there are any underlying errors during the import.

This is reproducible on Data Center: not yet

So far, this behaviour has only been observed on the SQLServer database, where, due to a certain DB constraint, the import job was not able to record the import results.

Steps to Reproduce

Have not been able to reproduce this in-house yet - However, the idea would be to create a large import that creates a LOT (>2000) of errors.

Expected Results

Assets Import result is always recorded:

2024-06-12 15:16:56,754 [insight-InsightThreadGroup-worker-thread-7] | <NAME_OF_IMPORT>>>> Finished import with id <ID> of type rlabs-import-type-<TYPE> at time YYYY-MM-DDThh:mm:ss.ms. Import took XX ms
2024-06-12 15:16:56,786 [insight-InsightThreadGroup-worker-thread-7] | com.riadalabs.jira.plugins.insight.services.imports.common.ImportResult@3b318283[
  status=FINISHED
  started=Wed Jun 12 15:16:55 UTC 2024
  ended=Wed Jun 12 15:16:56 UTC 2024
  executedType=XXXXX
  executedAsUserKey=xxxx
  objectSchemaId=X
  numberOfWorkers=XX
  importSourceId=X
  populatedObjectTypes=[CSV import, CSV import]
  moduleKey=rlabs-import-type-csv
  moduleVersion=1
  updateReferencesTimeInMs=XX
  onlyExecutedForObjectTypeIds=[]
  result=OK
  infoMessage=<null>
  errorMessages=<null>
  ObjectTypes : objectTypeResultMap={38=com.riadalabs.jira.plugins.insight.services.imports.common.ImportResult$ObjectTypeResult@1b5900b3[
  objectTypeName=CSV import
  objectTypeId=XX
  objectsUpdated=0
  objectsCreated=0
  objectsIdentical=27
  entriesInSource=27
  objectsFilteredWithIQL=0
  duplicateEntries=0
  emptyLabelEntries=0
  emptyExternalIdEntries=0
  objectsMissingUpdated=0
  objectsMissingDeleted=0
  errorMessages=<null>
  readExternalDataTimeInMs=16
  mapExternalDataTimeInMs=0
  iqlFilteringDataTimeInMs=0
  decidingActionsTimeInMs=16
  writeInsightDataTimeInMs=0
  postFunctionTimeInMs=1
  executionTimeInMs=33
  objectsWithUpdatedReferences=0
]}
]

Actual Results

You will see the import completed but it won't be followed up by the ImportResult in the insight-import.log:

2024-06-12 15:16:56,754 [insight-InsightThreadGroup-worker-thread-7] | <NAME_OF_IMPORT>>>> Finished import with id <ID> of type rlabs-import-type-<TYPE> at time YYYY-MM-DDThh:mm:ss.ms. Import took XX ms
......
.....
NO SUMMARY

On atlassian-jira.log it will record this exception Length of LOB data (XXXXXX) to be replicated exceeds configured maximum YYYYY. Use the stored procedure sp_configure to increase the configured maximum value for max text repl size option, which defaults to 65536. A configured value of -1 indicates no limit, other that the limit imposed by the data type.

2024-04-01 17:02:35,458+0000 insight-InsightThreadGroup-worker-thread-6773 WARN xxxx 1013x248445x1 1kje5e4 1.1.1.1,2.2.2.2 /rest/insight/1.0/import/start/60 [c.r.j.p.i.s.imports.common.DefaultImportService] Unable to create progress result
java.lang.reflect.UndeclaredThrowableException
	at com.sun.proxy.$Proxy9364.save(Unknown Source)
	at com.riadalabs.jira.plugins.insight.services.core.dal.impl.ProgressDalImpl.persistImportOTResult(ProgressDalImpl.java:255)
	at com.riadalabs.jira.plugins.insight.services.core.dal.impl.ProgressDalImpl.persistProgressResult(ProgressDalImpl.java:221)
	at com.riadalabs.jira.plugins.insight.services.progress.service.ResultRepositoryImpl.consumeResultAndClearInFlight(ResultRepositoryImpl.java:127)
	at com.riadalabs.jira.plugins.insight.services.progress.service.ResultRepositoryImpl.persistAndClearInFlight(ResultRepositoryImpl.java:118)
	at com.riadalabs.jira.plugins.insight.services.imports.common.DefaultImportService.doImportFromSource(DefaultImportService.java:768)
	at com.riadalabs.jira.plugins.insight.services.imports.common.DefaultImportService.lambda$_startImport$0(DefaultImportService.java:614)
	at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Length of LOB data (1028718) to be replicated exceeds configured maximum 512000. Use the stored procedure sp_configure to increase the configured maximum value for max text repl size option, which defaults to 65536. A configured value of -1 indicates no limit, other that the limit imposed by the data type.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1632)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:602)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:524)
	at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7418)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:247)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:222)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:473)
	at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:98)
	at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:98)
	at com.atlassian.jira.ofbiz.sql.PreparedStatementWrapper.executeUpdate(PreparedStatementWrapper.java:47)
	at com.atlassian.jira.diagnostic.connection.DiagnosticPreparedStatement.lambda$executeUpdate$7(DiagnosticPreparedStatement.java:69)
	at com.atlassian.diagnostics.internal.platform.monitor.db.DefaultDatabaseDiagnosticsCollector.recordExecutionTime(DefaultDatabaseDiagnosticsCollector.java:91)
	at com.atlassian.jira.diagnostic.connection.DatabaseDiagnosticsCollectorDelegate.recordExecutionTime(DatabaseDiagnosticsCollectorDelegate.java:62)
	at com.atlassian.jira.diagnostic.connection.DiagnosticPreparedStatement.executeUpdate(DiagnosticPreparedStatement.java:69)
	at net.java.ao.ForwardingPreparedStatement.executeUpdate(ForwardingPreparedStatement.java:45)
	at net.java.ao.ParameterMetadataCachingPreparedStatement.executeUpdate(ParameterMetadataCachingPreparedStatement.java:10)
	at net.java.ao.EntityProxy.save(EntityProxy.java:574)
	at net.java.ao.EntityProxy.invoke(EntityProxy.java:116)
	... 12 more

Followed by a NullPointerException as well:

2024-04-01 17:02:35,465+0000 insight-InsightThreadGroup-worker-thread-6773 ERROR xxxx 1013x248445x1 1kje5e4 1.1.1.1,2.2.2.2 /rest/insight/1.0/import/start/60 [c.r.j.p.i.s.analytics.service.ImportExportAnalyticsServiceImpl] Could not publish import result analytics
java.lang.NullPointerException
	at com.riadalabs.jira.plugins.insight.services.analytics.service.ImportExportAnalyticsServiceImpl.publishImportFinishedEvent(ImportExportAnalyticsServiceImpl.java:40)
	at com.riadalabs.jira.plugins.insight.services.imports.common.DefaultImportService.finishProgress(DefaultImportService.java:1599)
	at com.riadalabs.jira.plugins.insight.services.imports.common.DefaultImportService.doImportFromSource(DefaultImportService.java:776)
	at com.riadalabs.jira.plugins.insight.services.imports.common.DefaultImportService.lambda$_startImport$0(DefaultImportService.java:614)
	at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

The import job retains the import result on the the DB tables where it tried to write a JSON file which was bigger than the SQLServer's configured max text repl size. This typically may happen on a very large import where there were a lot of errors occurred that made the JSON file massive and it fails to write to the DB table due to this settings.

Workaround

Possible workaround until a permanent fix is introduced -

Consult your SQLServer DBA and you can set the max text repl size to -1
- https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/configure-the-max-text-repl-size-server-configuration-option?view=sql-server-ver16

mentioned in: Page Loading...; Page Loading...

Details

Description

Issue Summary

Steps to Reproduce

Expected Results

Actual Results

Workaround

Attachments

Issue Links

Forms

Activity

People

Dates