The various filtering options (statement contexts, method contexts, exclusions, etc) do not seem to be applied consistently across the various clover2 maven goals: clover2:clover, clover2:check: clover2:log. The discrepancies between clover2:clover and clover2:check essentially render clover2:check useless as it cannot be relied upon for accurate threshold reporting.

The attached maven project demonstrates the following use cases:
1. A method covered by tests
2. A method not covered by tests
3. A property method excluded via the property method context
4. A constructor method excluded via the constructor method context
5. A method filtered using a custom method context
6. A statement filtered using a custom statement context
7. A file filtered using the "exclusion" configuration option

In order to execute the example, unzip the attached zip file and run the following:

mvn clean clover2:setup test clover2:aggregate clover2:clover clover2:check clover2:log

Things to note:

Clover2:clover generates correctly. Coverage is at 50% and 71.4% of the code is correctly filtered out.

Clover2:check reports coverage at 66.7%. Now in this case clover2:check is reporting coverage higher than clover2:clover. I'm not sure how it's getting there, but in my production examples the opposite tends to be the case: clover2:check reports lower coverage than clover2:clover.

Clover2:log reports coverage at 33.3%:

[INFO] Clover Coverage Report
Coverage Timestamp: Fri Mar 02 12:26:09 EST 2012

Coverage Overview -
Coverage:-
Methods: 2/7 (28.6%)
Statements: 4/12 (33.3%)
Branches: 1/2 (50%)
Total: 33.3%
Complexity:-
Avg Method: 1.1428572
Density: 0.6666667
Total: 8

If I change my maven execution to the following:

mvn clean clover2:instrument test clover2:aggregate clover2:clover clover2:check clover2:log

where clover2:setup is swapped out for clover2:instrument, the coverage check matches the report: 50%.

As you can see, none of these tools seems to be reporting the same thing. You might say "Leo, why not use clover2:instrument so at least clover2:clover and clover:2check" match? Well, there's a problem with clover2:instrument as well: http://forums.atlassian.com/thread.jspa?threadID=22387

I've been using clover for some time now (~5+ years). Your main competitive advantage over the other guys (jacoco, cobertura, emma, etc) is your advanced filtering and reporting capabilities. The frustrating thing is that in these 5 years none of these features have ever seemed to work consistently across maven goals nor in integration with other tools like STS and Sonar.

Please make it a priority to ensure that these features are enabled consistently and tested for as part of every release. You know, ensure you have proper code coverage on your code coverage product.

Please feel free to integrate my example project into your automated regression suite!

Thanks!