Uploaded image for project: 'Clover'
  1. Clover
  2. CLOV-1256

as a developer I'd like to instrument tests written in the Spock framework

    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      We use Clover for both Java and Groovy.

      Unfortunately, Clover does not understand the tests we write in Spock.

      So a test like this:

      class OpsWiseJobQueryTestSpec extends AbstractTestSpec {
      	OpsWiseJobQuery wait
      	static File existingFileMarker = File.createTempFile("opswiseJobQuery", ".txt")
      	
      	void setup() {
      		wait = new OpsWiseJobQuery()
      		wait.manifestDir = existingFileMarker.getParentFile()
      		existingFileMarker.createNewFile()
      	}
      	
      	void cleanup() {
      		existingFileMarker.delete()
      	}
      	
      	def "should work with completed job"() {
      		expect:
      			state == wait.getJobState(manifestFilename)
      		where:
      			state						|	manifestFilename
      			OpsWiseJobState.WAITING 	|   "non-existing-name.whatever"
      			OpsWiseJobState.DONE		|	existingFileMarker.getName()
      	}	
      }
      

      Ends up with a coverage report like this:

      Class 	Tests 	Fail 	Error 	Time (secs) 	% Tests Success
      OpsWiseJobQueryTestSpec 	12 	0 	0 	0,026 	100% 	
       
      Tests 	Started 	Status 	Time (secs) 	Message
      OpsWiseJobQueryTestSpec. 	12 mar 10:32:50 	PASS 	0 	 
      OpsWiseJobQueryTestSpec.$spock_feature_1_0 	12 mar 10:32:49 	PASS 	0,001 	 
      OpsWiseJobQueryTestSpec. 	12 mar 10:32:49 	PASS 	0 	 
      OpsWiseJobQueryTestSpec.setup 	12 mar 10:32:49 	PASS 	0,009 	 
      OpsWiseJobQueryTestSpec. 	12 mar 10:32:50 	PASS 	0,001 	 
      OpsWiseJobQueryTestSpec.$spock_feature_1_0 	12 mar 10:32:50 	PASS 	0 	 
      OpsWiseJobQueryTestSpec.cleanup 	12 mar 10:32:50 	PASS 	0,01 	 
      OpsWiseJobQueryTestSpec.setup 	12 mar 10:32:50 	PASS 	0,001 	 
      OpsWiseJobQueryTestSpec.cleanup 	12 mar 10:32:50 	PASS 	0,001 	 
      OpsWiseJobQueryTestSpec. 	12 mar 10:32:49 	PASS 	0,002 	 
      OpsWiseJobQueryTestSpec. 	12 mar 10:32:49 	PASS 	0,001 	 
      OpsWiseJobQueryTestSpec. 	12 mar 10:32:49 	PASS 	0 	  
      

      I would expect to see just a single test passed.

            [CLOV-1256] as a developer I'd like to instrument tests written in the Spock framework

            Hey, in case you didn't notice - the Clover 3.3 has been released: http://www.atlassian.com/software/clover/download
            Have fun with Clover & Spock!

            Marek Parfianowicz added a comment - Hey, in case you didn't notice - the Clover 3.3 has been released: http://www.atlassian.com/software/clover/download Have fun with Clover & Spock!

            Update: the Clover 3.2.2 has been released, but it's has a Spock feature disabled. It will be officially released in Clover 3.3.0. Therefore, in order to check how Spock support works, please use the Clover 3.3.0-SNAPSHOT. I intent to add also a support for JUnit4 @Parameterized annotation in 3.3.0.

            See https://confluence.atlassian.com/display/CLOVER/Clover+Road+Map

            Marek Parfianowicz added a comment - Update: the Clover 3.2.2 has been released, but it's has a Spock feature disabled . It will be officially released in Clover 3.3.0. Therefore, in order to check how Spock support works, please use the Clover 3.3.0-SNAPSHOT. I intent to add also a support for JUnit4 @Parameterized annotation in 3.3.0. See https://confluence.atlassian.com/display/CLOVER/Clover+Road+Map

            Marek Parfianowicz added a comment - - edited

            > but I don't see it picking up any spock tests

            Do you see them instrumented at all (present in the HTML report, source lines coloured in red)? Or it's instrumented but you don't see test results? Or per-test coverage?

            Do you have clover.jar in the compilation classpath for groovyc?

            Marek Parfianowicz added a comment - - edited > but I don't see it picking up any spock tests Do you see them instrumented at all (present in the HTML report, source lines coloured in red)? Or it's instrumented but you don't see test results? Or per-test coverage? Do you have clover.jar in the compilation classpath for groovyc?

            Clover looks for Spock's annotations for specification classes and feature methods (you don't have them declared in the code, but they are being added by the Spock framework during AST transformation during compilation by groovyc), so the class name is not important.

            > Does clover only look in the src/test/groovy directory?

            How is your pom.xml configured? Where do you keep Groovy files? There are few recommendations about it. See:

            https://confluence.atlassian.com/display/CLOVER/Compiling+Groovy+with+GMaven+plugin

            https://confluence.atlassian.com/display/CLOVER/Compiling+Groovy+with+Groovy+Eclipse+Plugin

            https://bitbucket.org/atlassian/maven-clover2-plugin/src/3020f47cc36c08e08d1d139883ab891958613db2/src/it/?at=default

            Marek Parfianowicz added a comment - Clover looks for Spock's annotations for specification classes and feature methods (you don't have them declared in the code, but they are being added by the Spock framework during AST transformation during compilation by groovyc), so the class name is not important. > Does clover only look in the src/test/groovy directory? How is your pom.xml configured? Where do you keep Groovy files? There are few recommendations about it. See: https://confluence.atlassian.com/display/CLOVER/Compiling+Groovy+with+GMaven+plugin https://confluence.atlassian.com/display/CLOVER/Compiling+Groovy+with+Groovy+Eclipse+Plugin https://bitbucket.org/atlassian/maven-clover2-plugin/src/3020f47cc36c08e08d1d139883ab891958613db2/src/it/?at=default

            I'm running a Grails application and in that framework, all the Spock tests are placed in <appName>/test/unit or <appName>/test/integration

            Bryan Campbell added a comment - I'm running a Grails application and in that framework, all the Spock tests are placed in <appName>/test/unit or <appName>/test/integration

            That's interesting Jason, I thought the guidance from Spock was to name them ______spec.

            Bryan Campbell added a comment - That's interesting Jason, I thought the guidance from Spock was to name them ______spec.

            I have been able to run this successfully, but I don't see it picking up any spock tests (we name them ____SpockTest, so the standard pattern should pick them up). Does clover only look in the src/test/groovy directory?

            Jason Pelzer added a comment - I have been able to run this successfully, but I don't see it picking up any spock tests (we name them ____SpockTest, so the standard pattern should pick them up). Does clover only look in the src/test/groovy directory?

            Marek Parfianowicz added a comment - maven-clover2-plugin-3.3.0-SNAPSHOT.jar

            OK, I will need that maven plugin, or instructions on how to use the legacy plugin with the new core.

            I added 3.3.0-SNAPSHOT to the dependencies list for the 3.2.0 plugin, but got a stacktrace during site:site.

            Jason Pelzer added a comment - OK, I will need that maven plugin, or instructions on how to use the legacy plugin with the new core. I added 3.3.0-SNAPSHOT to the dependencies list for the 3.2.0 plugin, but got a stacktrace during site:site.

            Will try it today. We do use Maven, but I'll try to use the older version before asking for a special build. Thanks!

            Jason Pelzer added a comment - Will try it today. We do use Maven, but I'll try to use the older version before asking for a special build. Thanks!

              mparfianowicz Marek Parfianowicz
              365070a405d9 Jesper Skov
              Votes:
              5 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: