-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Low
-
Component/s: Chat - Chat Response Relevance
-
None
-
Severity 2 - Major
Issue Summary
Rovo in Jira Cloud, when used on a site with Xray Test Management, responds to a natural‑language request by stating that it will create Xray Test Executions and manual tests from a release note and will “get back” with links, but no Xray entities are created and no follow‑up summary is sent. This results in Rovo hallucinating actions and outcomes that never actually occur.
Steps to Reproduce
- In a Jira Cloud site with Rovo enabled and Xray installed (no custom Rovo–Xray integration), open Rovo in Jira.
- Let Rovo respond until it says it will create the Test Execution/tests and get back with a summary.
- In Jira/Xray, check if any new Test Execution or manual Xray Tests were created.
- Check the Rovo conversation for a final message with real links to created Xray issues.
Expected Results
- If Rovo cannot actually create Xray entities on this site:
- It should not say it will create Test Executions or manual tests or “get back” with links.
- It should only describe or draft a test plan and clearly say that Xray entities must be created manually or via other automation.
- If/when Rovo does support creating Xray entities: When it says it will create a Test Execution and tests and send a summary, those items should be created in Xray and a final message with real links should be posted.
Actual Results
- In the conversation, Rovo says things like:
- “i will proceed with the structed creatoin of all manual x-ray tests…”
- “i will get back soon as the creation is finalized with the complete summary and all the links…”
- “i’ll start creating all the manual x-ray tests… and then corresponding test execution.”
- In Jira/Xray:
- No Test Execution is created.
- No manual Xray Tests are created.
- No final summary with working links is posted.
- The user is left believing actions were or will be performed, but nothing happens.
Workaround
Currently there is no known workaround for this behaviour. A workaround will be added here when available