Issue Summary

During a full Enterprise Insights workflow run, the ETL Glue job became stuck processing a single object and did not progress for several hours.

Steps to Reproduce

1. Wait for a full Enterprise Insights workflow for the affected environment.
2. Observe the ETL Glue job (*-dw-etl-pipeline) during the story object processing:
   • Job progress reaches x/n objects and then does not progress for several hours.
3. After several hours of no progress, manually restart the ETL job (job-level restart, not re‑running the higher-level workflow).
4. Wait for ETL to complete.
5. Observe whether the *-dw-export job is automatically triggered when ETL finishes.

Expected Results

Job execution / progress behavior

• A full ETL workflow run should either:
• Complete within a predictable time window (e.g., 3–4 hours for this environment)

Workflow chaining behavior

• When the ETL step in the full workflow is restarted (even after a stall), the workflow orchestration should:
  • Preserve the expected chaining, and
  • Automatically trigger the export (*-dw-export) once ETL successfully completes.

Actual Results

Job execution / progress

• • The -dw-etl-pipeline job remained in a “running” state for *8+ hours while stuck on the story object at x/n.

Workflow chaining

• • After manually restarting the ETL job:
    • ETL completed successfully

• • • The associated export job did not start automatically.
  • The export (*-dw-export) had to be manually triggered
  • This resulted in additional delay to parquet availability

Workaround

Currently there is no known workaround for this behavior. A workaround will be added here when available