JIRA
  1. JIRA
  2. JRA-7688

Provide integration to "Test Case Management" databases/tools

    Details

    • Type: New Feature New Feature
    • Status: Resolved (View Workflow)
    • Priority: Major Major
    • Resolution: Answered
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Deployment
    • Labels:
      None

      Description

      one of the important tools in the eco-center of software development and issue tracking is test case management.

      Tools such as TestLink, TestMaster, Mercury TestDirector provide a database with the relevant functionality.

      Integration to the issue tracker provides a link between tests and the issues they created, are blocked on, etc.

      JIRA should have a generic integration to such tools, and specific easy integration to 1 or 2 leading tools.

      Best would be if JIRA can provide such a database on its own. Its not a rocket science field, and integrated support can be very interesting.

      see http://rndmgmttrenches.blogspot.com/2005/08/building-test-case-management-solution.html for my take on the issue.

        Issue Links

          Activity

          Hide
          Jeff Turner added a comment -

          If you have suggestions for how the integration should work, we're all ears. Quite a lot can be accomplished with the custom fields and workflow post-functions (eg. see the Perforce integration - http://confluence.atlassian.com/display/JIRAEXT/Perforce+JIRA+Plugin)

          Show
          Jeff Turner added a comment - If you have suggestions for how the integration should work, we're all ears. Quite a lot can be accomplished with the custom fields and workflow post-functions (eg. see the Perforce integration - http://confluence.atlassian.com/display/JIRAEXT/Perforce+JIRA+Plugin )
          Hide
          yuval yeret added a comment -

          I don't have enough experience with either JIRA or the other side of the integration to really suggest a design.

          I can try to elaborate on the requirements a bit:

          • In a defect provide a link to a test case the defect was found under (this helps verify you have coverage for bugs as well in your test plan, and helps the developers who are looking at the bug understand the scenario clearly. No need to mention generic steps and descriptions of scenarios when referring to a known test case)
          • In a feature or other issue type provide a link to a test case the defect should be veified under (again helps with coverage).
          • In a defect provide a link to a test run result that caused the defect. This provides a pointer to more information about the scenario of the bug, the test results, logs, etc.
          • For a test case / test script provide a link to a feature/bug/other issue which is blocking that test case. (dependencies). This allows reporting what percentage of the test plan is blocked by open issues.
          • Notify upon test plans that become unblocked since bugs have been resolved.
          Show
          yuval yeret added a comment - I don't have enough experience with either JIRA or the other side of the integration to really suggest a design. I can try to elaborate on the requirements a bit: In a defect provide a link to a test case the defect was found under (this helps verify you have coverage for bugs as well in your test plan, and helps the developers who are looking at the bug understand the scenario clearly. No need to mention generic steps and descriptions of scenarios when referring to a known test case) In a feature or other issue type provide a link to a test case the defect should be veified under (again helps with coverage). In a defect provide a link to a test run result that caused the defect. This provides a pointer to more information about the scenario of the bug, the test results, logs, etc. For a test case / test script provide a link to a feature/bug/other issue which is blocking that test case. (dependencies). This allows reporting what percentage of the test plan is blocked by open issues. Notify upon test plans that become unblocked since bugs have been resolved.
          Hide
          yuval yeret added a comment -

          I don't have enough experience with either JIRA or the other side of the
          integration to really suggest a design.

          I can try to elaborate on the requirements a bit:

          • In a defect provide a link to a test case the defect was found under
            (this helps verify you have coverage for bugs as well in your test plan, and
            helps the developers who are looking at the bug understand the scenario
            clearly. No need to mention generic steps and descriptions of scenarios when
            referring to a known test case)
          • In a feature or other issue type provide a link to a test case the
            defect should be veified under (again helps with coverage).
          • In a defect provide a link to a test run result that caused the
            defect. This provides a pointer to more information about the scenario of
            the bug, the test results, logs, etc.
          • For a test case / test script provide a link to a feature/bug/other
            issue which is blocking that test case. (dependencies). This allows
            reporting what percentage of the test plan is blocked by open issues.
          • Notify upon test plans that become unblocked since bugs have been
            resolved.
          Show
          yuval yeret added a comment - I don't have enough experience with either JIRA or the other side of the integration to really suggest a design. I can try to elaborate on the requirements a bit: In a defect provide a link to a test case the defect was found under (this helps verify you have coverage for bugs as well in your test plan, and helps the developers who are looking at the bug understand the scenario clearly. No need to mention generic steps and descriptions of scenarios when referring to a known test case) In a feature or other issue type provide a link to a test case the defect should be veified under (again helps with coverage). In a defect provide a link to a test run result that caused the defect. This provides a pointer to more information about the scenario of the bug, the test results, logs, etc. For a test case / test script provide a link to a feature/bug/other issue which is blocking that test case. (dependencies). This allows reporting what percentage of the test plan is blocked by open issues. Notify upon test plans that become unblocked since bugs have been resolved.
          Hide
          Anton Mazkovoi [Atlassian] added a comment -

          Hi,

          Thank you for providing the reuiqrements. Please let me address each one in turn for possible ways to implement it in JIRA:

          > In a defect provide a link to a test case the defect was found under (this helps verify you have coverage for bugs as well in your test plan, and helps the evelopers
          > who are looking at the bug understand the scenario clearly. No need to mention generic steps and descriptions of scenarios when referring to a known test case)

          As far as I can tell the the test cases are kept in an external system which is accessible with a web browser. If so, the link can be provided in a URL Custom Field. The custom field can be setup to appear only for defect (bug) issues, for example. For more information on custom fields please see:
          http://www.atlassian.com/software/jira/docs/latest/customfields/overview.html

          Also, please note that JIRA will automatically detect hyperlinks in all long text fields, so the links can be simply inserted in the description of the issue or in a free text custom field.

          > In a feature or other issue type provide a link to a test case the defect should be veified under (again helps with coverage).

          That can be another URL Custom Field (called e.g. Verified Test Case) or another simply link in a text field.

          > In a defect provide a link to a test run result that caused the defect. This provides a pointer to more information about the scenario of the bug,
          > the test results, logs, etc.

          As above.

          > For a test case / test script provide a link to a feature/bug/other issue which is blocking that test case. (dependencies). This allows reporting what percentage of the > test plan is blocked by open issues.

          If a test case is actually an issue in JIRA, Issue Linking can be used to represent dependencies. For more information on issue linking please see:
          http://www.atlassian.com/software/jira/docs/latest/issuelinking.html

          It is then possible to build custom workflow conditions to e.g. restrict resolution of issues while dependent issues are still open. Custom reports can also be built to show the dependency graph of issues. While these conditions and reports are not shipped with JIRA at the moment, information about how to implement them can be found here:
          http://confluence.atlassian.com/pages/viewpage.action?pageId=11764
          http://confluence.atlassian.com/pages/viewpage.action?pageId=11465

          > Notify upon test plans that become unblocked since bugs have been resolved.

          In JIRA the easiest way to notify people or external system is to create a Listener that will od the required work on a particulr reports. Listeners are described here:
          http://www.atlassian.com/software/jira/docs/latest/listeners.html

          Please let me know what you think about the above.

          Building all of these as a single plugin for JIRA would be very cool indeed. However, given our current work load it might be some time until we will be able to make this plugin a reality. I hope that JIRA already exposes all the right hooks for the integration to be built. Of course, you are more than welcome to implement this integration in-house.

          Thanks,
          Anton

          Show
          Anton Mazkovoi [Atlassian] added a comment - Hi, Thank you for providing the reuiqrements. Please let me address each one in turn for possible ways to implement it in JIRA: > In a defect provide a link to a test case the defect was found under (this helps verify you have coverage for bugs as well in your test plan, and helps the evelopers > who are looking at the bug understand the scenario clearly. No need to mention generic steps and descriptions of scenarios when referring to a known test case) As far as I can tell the the test cases are kept in an external system which is accessible with a web browser. If so, the link can be provided in a URL Custom Field. The custom field can be setup to appear only for defect (bug) issues, for example. For more information on custom fields please see: http://www.atlassian.com/software/jira/docs/latest/customfields/overview.html Also, please note that JIRA will automatically detect hyperlinks in all long text fields, so the links can be simply inserted in the description of the issue or in a free text custom field. > In a feature or other issue type provide a link to a test case the defect should be veified under (again helps with coverage). That can be another URL Custom Field (called e.g. Verified Test Case) or another simply link in a text field. > In a defect provide a link to a test run result that caused the defect. This provides a pointer to more information about the scenario of the bug, > the test results, logs, etc. As above. > For a test case / test script provide a link to a feature/bug/other issue which is blocking that test case. (dependencies). This allows reporting what percentage of the > test plan is blocked by open issues. If a test case is actually an issue in JIRA, Issue Linking can be used to represent dependencies. For more information on issue linking please see: http://www.atlassian.com/software/jira/docs/latest/issuelinking.html It is then possible to build custom workflow conditions to e.g. restrict resolution of issues while dependent issues are still open. Custom reports can also be built to show the dependency graph of issues. While these conditions and reports are not shipped with JIRA at the moment, information about how to implement them can be found here: http://confluence.atlassian.com/pages/viewpage.action?pageId=11764 http://confluence.atlassian.com/pages/viewpage.action?pageId=11465 > Notify upon test plans that become unblocked since bugs have been resolved. In JIRA the easiest way to notify people or external system is to create a Listener that will od the required work on a particulr reports. Listeners are described here: http://www.atlassian.com/software/jira/docs/latest/listeners.html Please let me know what you think about the above. Building all of these as a single plugin for JIRA would be very cool indeed. However, given our current work load it might be some time until we will be able to make this plugin a reality. I hope that JIRA already exposes all the right hooks for the integration to be built. Of course, you are more than welcome to implement this integration in-house. Thanks, Anton
          Hide
          yuval yeret added a comment -

          sounds ok

          custom field is better for all of those in order to enable reports like
          "features/bugs without tests" or "tests that need to be rerun due to a
          certain release"

          The problem is when a test case is not in issue in JIRA as assumed above...
          If its in JIRA all of it will be easier, but then the relationships are much
          more complicated and not really supported out of the box by an Issue Tracker
          data model.

          It is then possible to build custom workflow conditions to e.g. restrict

          In general, the links are the easy part of the equation.
          The problem is when you need more information about the other side than the
          simple ID or existence. (e.g. whether a test case succeeded...)

          Building all of these as a single plugin for JIRA would be very cool indeed.

          I'll continue to think about this...

          Thanks,

          Show
          yuval yeret added a comment - sounds ok custom field is better for all of those in order to enable reports like "features/bugs without tests" or "tests that need to be rerun due to a certain release" The problem is when a test case is not in issue in JIRA as assumed above... If its in JIRA all of it will be easier, but then the relationships are much more complicated and not really supported out of the box by an Issue Tracker data model. It is then possible to build custom workflow conditions to e.g. restrict In general, the links are the easy part of the equation. The problem is when you need more information about the other side than the simple ID or existence. (e.g. whether a test case succeeded...) Building all of these as a single plugin for JIRA would be very cool indeed. I'll continue to think about this... Thanks,
          Show
          Benjamin Naftzger [Atlassian] added a comment - More comments here: http://forums.atlassian.com/thread.jspa?messageID=257222017
          Hide
          Ernest Wong [Atlassian] added a comment -
          Show
          Ernest Wong [Atlassian] added a comment - An interesting spec is posted by the JIRA community: http://confluence.atlassian.com/display/JIRACOM/JIRA+to+Mercury+TestDirector+bridge
          Hide
          Daoming Wang added a comment - - edited

          I think it is a very important feature. I need it.
          esp. a plugin linking to HP Quality Center is very good.

          Show
          Daoming Wang added a comment - - edited I think it is a very important feature. I need it. esp. a plugin linking to HP Quality Center is very good.
          Hide
          Brian Lane added a comment -

          There is a confluence page that discusses the integration of various test case management tools with JIRA.

          http://confluence.atlassian.com/display/JIRACOM/Integrate+JIRA+With+A+Test+Case+Management+Tool

          Regard,

          Brian

          Show
          Brian Lane added a comment - There is a confluence page that discusses the integration of various test case management tools with JIRA. http://confluence.atlassian.com/display/JIRACOM/Integrate+JIRA+With+A+Test+Case+Management+Tool Regard, Brian

            People

            • Assignee:
              Unassigned
              Reporter:
              yuval yeret
            • Votes:
              4 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: