Hey, in case you didn't notice - the Clover 3.3 has been released: http://www.atlassian.com/software/clover/download
Have fun with Clover & Spock!

Update: the Clover 3.2.2 has been released, but it's has a Spock feature disabled. It will be officially released in Clover 3.3.0. Therefore, in order to check how Spock support works, please use the Clover 3.3.0-SNAPSHOT. I intent to add also a support for JUnit4 @Parameterized annotation in 3.3.0.

See https://confluence.atlassian.com/display/CLOVER/Clover+Road+Map

Clover looks for Spock's annotations for specification classes and feature methods (you don't have them declared in the code, but they are being added by the Spock framework during AST transformation during compilation by groovyc), so the class name is not important.

> Does clover only look in the src/test/groovy directory?

How is your pom.xml configured? Where do you keep Groovy files? There are few recommendations about it. See:

https://confluence.atlassian.com/display/CLOVER/Compiling+Groovy+with+GMaven+plugin

https://confluence.atlassian.com/display/CLOVER/Compiling+Groovy+with+Groovy+Eclipse+Plugin

https://bitbucket.org/atlassian/maven-clover2-plugin/src/3020f47cc36c08e08d1d139883ab891958613db2/src/it/?at=default

I'm running a Grails application and in that framework, all the Spock tests are placed in <appName>/test/unit or <appName>/test/integration

That's interesting Jason, I thought the guidance from Spock was to name them ______spec.

I have been able to run this successfully, but I don't see it picking up any spock tests (we name them ____SpockTest, so the standard pattern should pick them up). Does clover only look in the src/test/groovy directory?

maven-clover2-plugin-3.3.0-SNAPSHOT.jar

OK, I will need that maven plugin, or instructions on how to use the legacy plugin with the new core.

I added 3.3.0-SNAPSHOT to the dependencies list for the 3.2.0 plugin, but got a stacktrace during site:site.

Will try it today. We do use Maven, but I'll try to use the older version before asking for a special build. Thanks!

By the way: two Clover releases were merged into one release, so the 3.2.1 was renamed to the 3.3.0. A stable (non-snapshot) 3.3.0 version should be available in January. I hope that using SNAPSHOT is not a problem for you. I'd like to notice that snapshots are actually quite stable, because they're being deployed to Atlassian Maven repository only if all JUnits have passed.

Hi Jason,

Spock framework support is practically complete (all features are done, I'm working on some extra tests and polishing details).

You can try it out already - use the latest 3.3.0-SNAPSHOT build from:

• https://maven.atlassian.com/public-snapshot/com/cenqua/clover/clover/3.3.0-SNAPSHOT/

It's just a Clover core, please let me know if you need a maven or grails plugin as well (I guess you can just use older maven/grails version with this one).

Cheers
Marek

TODO: document on supported platforms:

• minimum Spock version: spock-core-0.6
• for spock-core-0.6 and groovy 1.7 @Unroll with selectors (like "test #foo.bar") do not work - no matching constructor

Looking forward to it, thanks very much, Marek!

Hi Jason,

Indeed, we had a delay with Clover 3.2.0. I estimate that 3.2.1 will be released about 3-4 weeks later than originally planned (there were few things added to the scope). However, I will implement the Spock support as first, so you'll be able to get an early preview in 3.2.1-SNAPSHOT version.

Cheers
Marek

Since Clover 3.2 came out yesterday, I'm guessing 3.2.1's date is somewhat further out than next week... Any specific date targeted?

That is awesome news. Thanks Marek for helping to get this prioritized.

Great!

Update:

• the Clover's test optimization feature for Spock framework will be handled separately - see linked issue CLOV-1364 (we don't want to force you to wait too long for a basic Spock support)
• we aim to release Clover 3.2.1 at the end of October

Hi Jason and Bryan,

Our manager decided to prioritize this feature. It was moved to the release 3.2.1 (it should be released before end of this year).

Cheers
Marek

Thank you very much for your feedback. I will inform our development manager about your concerns.

Have to agree with Bryan, 3.2.6 is a long way off. We already have a Clover license, but if support is 2+ years off (judging by previous clover release cycle), it seems silly to keep up the contract. 80% of our tests have already been migrated to Spock, and 100% of our new tests are specs. Bummer!

Groovy is not supported in Clover-for-IDEA and Clover-for-Eclipse. Groovy code can be compiled using Clover-for-Ant / Maven / Grails.

We are already using several Atlassian products and were looking at evaluating Clover. Unfortunately the lack of Spock integration is preventing us from adopting it at this time. I'll watch this issue for progress, but 3.2.6 seems a long way off. And I'm sure there will be a delay after that in getting the Grails and IntelliJ plugins updated.

Spock is a main test framework for Grails, so increasing priority to Major and scheduling for Clover 3.2.6 ("Groovy / Grails / Gradle")

Test optimization with a Spock framework requires change how FileInfo coverage data is calculated. As far as I have analysed, a case is as follows:

The Spock test calls class Foo. Both sources files are compiled. Spock and Foo gets it's range of indexes in hitCount table.
First test optimization runs. Optimizer runs everything as it has no snapshot. Test snapshot is written and contains mapping like "Foo.groovy -> Spock.$spock_feature_0_0"

Now the Foo class is changed and recompiled. It gets new range of indexes in Clover's database. Spock test is NOT recompiled, because all test methods are defined as "def" and (my guess) Groovy postpones evaluation of the code to the runtime.

Consequence? Second optimized test runs. Test optimizer walks through all tests and finds "Spock.$spock_feature_0_0" as related with Foo. If finds that checksums are different and decides to execute it. Now test optimization snapshot is being updated after tests. However, the test optimizer is not able to match hit counts from the test execution to the index range of new Foo class, because new the Foo class has entirely new range of indexes. It does not update information about file checksum and timestamp.

Third optimized test runs. Situation repeats. Test is executed, but snapshot is not updated accordingly.

Workaround:

Delete Clover database and optimization snapshot and recompile project after every N runs.

Possible fix:

Snapshot.calcHits() shall get set of indexes for file (FileInfo) as a sum of ranges from all instrumentation sessions. Thanks to this, it should be able to go through build history and find proper file mapping.

Alternative: rebuild Spock test class if any dependent class was changed? It would get new range of indexes...

There is a workaround allowing to partially integrate Clover with Spock framework. There are two things to be done.

First, it's necessary to override the Clover's default test detector and provide custom class/method pattern for Spock tests. This is necessary to ensure that:

• Spock classes will be listed on the 'Test' source tab in HTML report
• only def "some test name" methods from Spock classes will be instrumented as test methods

clover {
    setuptask = { ant, binding, plugin ->
        ant.'clover-setup'(initstring: "target/clover/db/common.db") {
            ant.fileset(dir: "grails-app", includes: "**/domain/**, **/controllers/**, **/services/**, **/taglib/**, **/utils/**") { }
            ant.fileset(dir: "src", includes: "**/*.groovy, **/*.java") { }
            ant.testsources(dir: "test") {
                ant.or {
                    ant.testclass(name: '.*Spec') {    // Spock tests
                        // only  'def "some test name"' will be recognized as a test method
                        ant.testmethod(returntype: 'def', name: '$spock_feature.*') 
                    }
                    ant.testclass(name: '.*Tests')     // JUnit tests
                }
            }
            ant.profiles {
                // this is for larger Grails projects
                ant.profile(name: "default", coverageRecorder: "SHARED")  
            }
        }
    }
    ...

Second, it's necessary to disable Clover's built-in test result collection mechanism and point to JUnit-report XML files produced by Spock (and other test frameworks, if used).

    ...
    reporttask =  { ant, binding, plugin ->
        ant.delete(dir: "target/clover/report")
        ant.'clover-report'(initstring: "target/clover/db/common.db") {
            ant.current(outfile: "target/clover/report") {
                ant.format(type: "html")
                // use test results from XML files
                ant.testresults(dir: 'target/test-reports', includes: 'TEST-*.xml')   
            }
        }
    }
}

I couldn't make that work properly. It still showed closure/innerclass results, even though these do not appear in the XML unit test reports.

Unfortunately, I don't have time to look more at it for the next couple of weeks at least.
But I still think it would be nice if Clover had native support Spock.

Hello Jesper, thank you for raising a feature request.

Does Spock framework generates test results in JUnit-compatible XML format? I guess so... If yes, then you could try using the <clover-report>/<current>/<testsources>. Using the <testsources> an HTML report will not contain test results recorded by Clover but the ones provided by you.