Uploaded image for project: 'Clover'
  1. Clover
  2. CLOV-94

Clover incorrectly reports failing Instinct specifications when they expect an exception

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Medium Medium
    • 2.1.0
    • 2.0.2
    • Documentation
    • None

    • Linux duke 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007 i686 GNU/Linux

      Clover reports a "failure" when running Instinct specifications that expect an exception. Instinct does not propagate the exception (perhaps JUnit 4 does), and instead handles it during specification running.

      Here's an example of what causes the failure, a screenshot is attached shoing how this manifests in the HTML report).

      @Specification(expectedException = IllegalArgumentException.class, withMessage = "Population cannot be chunked with chunk size (-1) less than 1")
public void shouldNotAcceptNegativeChunkSize() {
    new PopulationToChunkMapperImpl(-1);
}

      Changing the code to the following "works" (i.e. doesn't cause Clover to fail).

      public void shouldNotAcceptNegativeChunkSize() {
    try {
        new PopulationToChunkMapperImpl(-1);
        throw new RuntimeException("Expected exception not thrown");
    } catch (IllegalArgumentException e) {
        // all good
    }
}

      Clover is enabled/configured as follows:

      <clover-setup initstring="${coverage.db}" tmpdir="${coverage.results.dir}/tmp" flushpolicy="threaded"
        flushinterval="${coverage.flush.interval}">
    <fileset refid="main.covered.fileset"/>
    <testsources dir="${spec.src.dir}">
        <testclass>
            <testmethod annotation="Specification"/>
        </testclass>
    </testsources>
</clover-setup>

      Maybe it's got something to do with how clover is configured. For example Instinct will also run spec methods (analogous to JUnit test methods) beginning with "should" and "must" however these do not appear to work correctly in all circumstances.

      This may be related to this issue: http://jira.atlassian.com/browse/CLOV-90.

        1. CloverFailingSpecification.png
          CloverFailingSpecification.png
          252 kB

            [CLOV-94] Clover incorrectly reports failing Instinct specifications when they expect an exception

            Owen made changes -
            Workflow Original: New Clover Workflow [ 897862 ] New: New Clover Workflow - Restricted [ 1474356 ]
            Piotr Swiecicki made changes -
            Workflow Original: Clover Workflow [ 896027 ] New: New Clover Workflow [ 897862 ]
            Piotr Swiecicki made changes -
            Workflow Original: reviewflow [ 119963 ] New: Clover Workflow [ 896027 ]

            Marek Parfianowicz added a comment - - edited

            I have documented Instinct and all other supported frameworks here: Clover+test+detection

            Marek Parfianowicz added a comment - - edited I have documented Instinct and all other supported frameworks here: Clover+test+detection
            Marek Parfianowicz made changes -
            Status Original: Resolved [ 5 ] New: Closed [ 6 ]
            Michael Studman (Inactive) made changes -
            Resolution New: Fixed [ 1 ]
            Status Original: To be reviewed [ 10026 ] New: Resolved [ 5 ]

            Michael Studman (Inactive) added a comment -

            Clover now understands how Instinct specifies expected exception types and reports correctly on these. We still need to do more work to make sure Clover also tests the expected exception message if specified. http://jira.atlassian.com/browse/CLOV-154

            Michael Studman (Inactive) added a comment - Clover now understands how Instinct specifies expected exception types and reports correctly on these. We still need to do more work to make sure Clover also tests the expected exception message if specified. http://jira.atlassian.com/browse/CLOV-154
            Michael Studman (Inactive) made changes -
            Status Original: Implemented [ 10025 ] New: To be reviewed [ 10026 ]
            BrendanA made changes -
            Status Original: Open [ 1 ] New: Implemented [ 10025 ]
            Matt Quail (Inactive) made changes -
            Workflow Original: JIRA Bug Workflow v2 [ 117272 ] New: reviewflow [ 119963 ]

              npellow Nick
              e64e536854ef Tom Adams
              Affected customers:
              0 This affects my team
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: