Issue Summary

Some object types do not have all objects imported via object schema import due to SocketTimeoutException

Steps to Reproduce

To reproduce this issue, we need a schema with objects that result in large API response size.

1. Perform an object schema import from one instance to another.
2. Compare the number of objects in the source and in the imported instance.

Expected Results

All objects are created/updated as expected design without SocketTimeoutException

Actual Results

• Some object type might not have all objects imported with the exception below in the insight_import.log. There's no error anywhere on the UI indicating this.
  In 10.0.1, the issue persists but there's no SocketTimeoutException in the insight_import.log. The log shows "Completed reading data for configured object type id <id> in <time>ms" for the first paginated response.

  2024-08-16 08:06:25,827 [insight-InsightImportThreadGroup-worker-thread-2] | Error getting list of objects for: <objectTypeName>;<source_objectTypeID>, Error: Read timed out
  java.net.SocketTimeoutException: Read timed out
  	at java.base/java.net.SocketInputStream.socketRead0(Native Method)
  	at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:484)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:478)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
  	at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1459)
  	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1070)
  	at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
  	at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:292)
  	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351)
  	at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:789)
  	at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:724)
  	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1636)
  	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1541)
  	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
  	at com.riadalabs.jira.plugins.insight.services.imports.modules.json.JSONStreamReader.readJSONRemoteObject(JSONStreamReader.java:68)
  	at com.riadalabs.jira.plugins.insight.services.imports.modules.objectschema.ObjectSchemaImportDataReader.getJSONObject(ObjectSchemaImportDataReader.java:1047)
  	at com.riadalabs.jira.plugins.insight.services.imports.modules.objectschema.ObjectSchemaImportDataReader.getObjectList(ObjectSchemaImportDataReader.java:407)
  	at com.riadalabs.jira.plugins.insight.services.imports.modules.objectschema.ObjectSchemaImportDataReader.readData(ObjectSchemaImportDataReader.java:192)
  	at com.riadalabs.jira.plugins.insight.services.imports.modules.objectschema.ObjectSchemaImportModule.dataHolder(ObjectSchemaImportModule.java:72)
  	at com.riadalabs.jira.plugins.insight.services.imports.modules.objectschema.ObjectSchemaImportModule.dataHolder(ObjectSchemaImportModule.java:39)
  	at com.riadalabs.jira.plugins.insight.services.jira.module.ImportModuleDelegator.dataHolder(ImportModuleDelegator.java:56)
  	at com.riadalabs.jira.plugins.insight.services.imports.common.importjobprovider.ImportJobProvider.getImportDataHolder(ImportJobProvider.java:136)
  	at com.riadalabs.jira.plugins.insight.services.imports.common.importjobprovider.DataHolderFetcherJobProvider$DataHolderFetcher.fetchDataHolder(DataHolderFetcherJobProvider.java:99)
  	at com.riadalabs.jira.plugins.insight.services.imports.common.importjobprovider.DataHolderFetcherJobProvider$DataHolderFetcher.executeTask(DataHolderFetcherJobProvider.java:89)
  	at com.riadalabs.jira.plugins.insight.services.imports.common.importjobprovider.DataHolderFetcherJobProvider$DataHolderFetcher.executeTask(DataHolderFetcherJobProvider.java:73)
  	at com.riadalabs.jira.plugins.insight.services.core.multithreadservice.InsightServiceJob.call(InsightServiceJob.java:41)
  	at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  	at java.base/java.lang.Thread.run(Thread.java:829)
  

• The affected object type might only have the first 250 objects imported successfully. This is because the size of pagination used in the endpoint for this import is 250 objects per page. The first paginated request succeed but the second one failed with the timeout error. With debug enabled on the source instance (com.riadalabs.jira.plugins.insight.channel.web.api.rest.services.aql), we can see that calling page 2 took longer than 30 seconds and the timeout error occurred after 30 seconds is up.
  • Debug log on atlassian-jira.log of the source instance. The call for page 2 started at 09:55:29.

    2024-09-12 09:55:29,249+0000 http-nio-8080-exec-13 url: /jsm/rest/insight/1.0/iql/objects; user: admin DEBUG admin 595x237214x5 1ie0uge 10.227.31.73,172.50.0.2 /rest/insight/1.0/iql/objects [c.r.j.p.i.c.w.a.r.services.aql.AQLResource] findObjects (1726134929) by AQL: objectTypeId=1277, objectSchemaId: [], page: 2, orderByAttributeId: null, orderAsc: true, resultPerPage: 250, includeAttributes: true, includeTypeAttributes: false, includeExtendedInfo: false
    ...
    2024-09-12 09:56:18,197+0000 http-nio-8080-exec-13 url: /jsm/rest/insight/1.0/iql/objects; user: admin DEBUG admin 595x237214x5 1ie0uge 10.227.31.73,172.50.0.2 /rest/insight/1.0/iql/objects [c.r.j.p.i.c.w.a.r.services.aql.AQLResource] findObjects (1726134929) by AQL took 48947 ms
    

  • insight_import.log of the target instance. It timed out at 09:56:02 (~30 seconds).

    2024-09-12 09:56:02,244 [insight-InsightImportThreadGroup-worker-thread-5] | Error getting list of objects for: Services;1277, Error: Read timed out
    java.net.SocketTimeoutException: Read timed out
    

Workaround

Import the schema via zip file as per https://confluence.atlassian.com/servicemanagementserver/importing-and-exporting-object-schemas-1044784445.html

Versions Tested

Please confirm all versions that have been tested for this issue, and indicate whether the tested version is affected or not affected, below: