Hey Mark,

Yes it does work in initialize phase as follows:

<execution>
                        <phase>initialize</phase>
                        <goals>
                            <goal>instrument-test</goal>
                            <goal>check</goal>
                            <goal>clover</goal>
                        </goals>
                    </execution>

Thanks

Hi Aditya,

I think that this is caused by a fact that you're using the clover2:instrument instead of the clover2:setup goal. The clover2:instrument forks a custom build life cycle and in this build cycle it performs instrumentation in the 'validate' phase. Due to a fact that your 'add-source' goal is bound to the generate-sources phase, the build-helper-maven-plugin runs after Clover instead of before.

I suggest to use clover2:setup bounded to the validate or initialize phase (i.e. before the generate-sources one).

Cheers
Marek

Hi bbusjaeger, I intend to release Clover 4.0.1 at the end of next week, so in case I would have to code some fixes for this bug, I need to get a feedback from you on the Tuesday (26.08) the latest. I'd be very grateful if you could have a look at my questions posted in previous comments. Cheers!

One more thing: can you tell whether you're using a parallel compilation in Maven?

I'm asking because Clover does not support parallel builds - it fails when multiple processes are trying to write to the same clover.db file at the same time.

In case running 'mvn -X' will not reveal more details, then I suggest running the CloverInstr tool manually on your sources generated by Avro. Thanks to this we will check whether there is a problem with running an instrumentation (for instance, some code construct which Clover cannot correctly parse). In case it succeeds, then we'll know that it's a problem with the maven-clover2-plugin.

How to use CloverInstr:

https://confluence.atlassian.com/display/CLOVER/CloverInstr

Regarding the stack trace:

Caused by: org.apache.maven.plugin.MojoExecutionException: Clover has failed to instrument the source files in the [/home/bbusjaeger/dev/app/main/core/apex-lang/target/clover/src-instrumented] directory
at com.atlassian.maven.plugin.clover.internal.instrumentation.AbstractInstrumenter.instrumentSources(AbstractInstrumenter.java:198)

This is caused by a non-zero return code from CloverInstr:

// from maven-clover2-plugin code:

int result = CloverInstr.mainImpl(createCliArgs(filesToInstrument, outputDir));
        if (result != 0) {
            throw new MojoExecutionException("Clover has failed to instrument the source files "
                    + "in the [" + outputDir + "] directory");
        }

Could you please run your build with a debug logging (mvn -X) and check if there are more log messages? I expect to see one of these:

  *** ERROR: <error message>
  USAGE: com.atlassian.clover.CloverInstr [OPTIONS] PARAMS [FILES...]

 ... more lines with command line options ...

This can indicate an error in passing arguments between the clover2:instrument/clover2:setup and the CloverInstr program.

Could not initialise Clover: <error message>

Problem with running instrumentation. There may be various reasons, for example "Unable to create or load clover registry located at: ...".

Invalid or missing License. Please visit .... to obtain a valid license.

Problem with a license. Rather unlikely as you're successfully instrumenting other files.

Yes.

Yes.

The exception trace is pasted above. I can send a build log, but it didn't have more information. I am happy to run with an instrumented version that reports why "Clover has failed to instrument the source files".

We're using excludes + copyExcludedFiles to work-around the issue,

You have copyExcludedFiles=false, am I correct?

but it's quite brittle, as devs have to update the excludes whenever they add files in the source folder

Does it mean that it's impossible to define (general enough) exclusion pattern?

Anyone looking at the 'Clover has failed to instrument the source files' error in 4.0.0?

Please have a look at my previous comments. I'd love to investigate it further but I haven't received any POMs or build logs for analysis. So at the moment I assume/guess that it's a problem with order in which goals are called.

We're using excludes + copyExcludedFiles to work-around the issue, but it's quite brittle, as devs have to update the excludes whenever they add files in the source folder.

Anyone looking at the 'Clover has failed to instrument the source files' error in 4.0.0?

There's an option to not copy excluded files (https://docs.atlassian.com/maven-clover2-plugin/latest/setup-mojo.html#copyExcludedFiles). By default Clover copies excluded files to ensure that code will compile and run properly. But in some cases you can set it to false.

For what it's worth, I've tried using the <excludes /> configuration element, and it doesn't seem to work:

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <jdk>${compiler.java.level}</jdk>
        <licenseLocation>${maven.clover.licenseLocation}</licenseLocation>
        <excludes>
            <exclude>**/Test*.java</exclude>
            <exclude>**/*IT.java</exclude>
            <exclude>**/*IntegrationTest.java</exclude>
        </excludes>
    </configuration>
</plugin>

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:40 min
[INFO] Finished at: 2014-07-30T09:56:43-06:00
[INFO] Final Memory: 101M/374M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project my-app: Compilation failure: Compilation failure:
[ERROR] /Users/me/git/my-project/src/it/java/com/my/app/integration/IntegrationTest.java:[7,8] duplicate class: com.my.app.integration.IntegrationTest
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

At the time, we were generating the sources into src/main/java of the project.

I think this is related with an order in which clover2:setup and Avro goals are called. You have to call Avro first, so that Clover will be able see these sources, instrument them and next switch the src/main/java to target/clover/src-generated.

I eventually refactored the project to put its sources in target/generated-sources ... I used the build-helper-maven-plugin to add that folder as a source folder

I think this is the same issue as above - you have to add this target/generated-sources folder first and next call the clover2:setup.

Sorry, lost track of this thread.

Hi Joshua,

Are you talking about the Apache Avro (http://avro.apache.org/docs/current/)?

Could you attach one sample source file generated by Avro? I'd like to have a look at it's structure to see why Clover failed to instrument it.

Did you encounter instrumentation errors on any non-Avro source files?

Correct, I am talking about Apache Avro (sorry for not specifying). I can't provide the code that triggered the issue since it's proprietary, but I strongly suspect that it's not a problem instrumenting the code so much as how Clover was copying the source files.

At the time, we were generating the sources into src/main/java of the project. The error seemed to be occurring because Clover was, when compiling the code after copying it into its post-instrumentation staging directory in target/ was trying to treat the code in its staging directory and src/main/java as sources of code to be compiled - as a result, it would try to compile both src/main/java/com/example/Foo.java and target/clover2/blah/blah/com/example/Foo.java and would error due to a canonical name collision.

I eventually refactored the project to put its sources in target/generated-sources which temporarily alleviated this problem, but when I used the build-helper-maven-plugin to add that folder as a source folder to get our m2eclipse integration to add it as a source folder in Eclipse, this issue happened again, for the same reason (though due to source in target/generated-sources instead of src/main/java this time).

we get the same "Clover has failed to instrument the source files" error with 4.0.0

By the way: I have released Clover 4.0.0 few days ago, so the most probably I won't release Clover 3.3.1. Therefore, in order to use my bug fix, you have to update to Clover 4.

Hi Joshua,

Are you talking about the Apache Avro (http://avro.apache.org/docs/current/)?

Could you attach one sample source file generated by Avro? I'd like to have a look at it's structure to see why Clover failed to instrument it.

Did you encounter instrumentation errors on any non-Avro source files?

Can you confirm that the 'duplicate class exception' does not occur anymore? I'd like to know whether I shall reopen this bug or rather create a separate one for Avro files.

You wrote that you're deleting */avro/* files. Did you try another approach? For example:

• you could define exclusion pattern for clover2:setup (https://docs.atlassian.com/maven-clover2-plugin/latest/setup-mojo.html#excludes)

• as Avro files are generated, then you may need to change the order in which Avro and Clover goals are called - the clover2:setup must be called when all generated files are already present (otherwise Clover's directory scanner won't find them and won't add to list of excluded files; see similar example for Clover + JAXB generated files:
  • https://confluence.atlassian.com/display/CLOVERKB/Duplicate+class+errors+with+Clover+and+JAXB+or+JAXB2+plugin
  • https://confluence.atlassian.com/display/CLOVERKB/Duplicate+class+errors+with+Clover+and+jaxws-maven+plugin

Cheers
Marek

I ran into this issue with Avro-generated classes and test classes; given that I wasn't concerned with collecting coverage of the former nor the latter, I got around it by using a plugin configuration like so:

<plugin>
    <artifactId>maven-clean-plugin</artifactId>
    <executions>
        <execution>
            <id>delete-clover-avro-classes</id>
            <phase>process-sources</phase>
            <goals>
                <goal>clean</goal>
            </goals>
            <configuration>
                <excludeDefaultDirectories>true</excludeDefaultDirectories>
                <filesets>
                    <fileset>
                        <directory>${project.build.directory}/clover/</directory>
                        <includes>
                            <include>**/avro/**/*</include>
                        </includes>
                    </fileset>
                </filesets>
            </configuration>
        </execution>
    </executions>
</plugin>

It's not the best solution and obviously won't work for people who want coverage of classes that are in source directories added by the build-helper-maven-plugin, but it's a workaround that will, hopefully, work for some.

[ERROR] Failed to execute goal com.atlassian.maven.plugins:maven-clover2-plugin:3.3.1:setup (clover-instrument) on project apex-lang: Clover has failed to instrument the source files in the [/home/bbusjaeger/dev/app/main/core/apex-lang/target/clover/src-instrumented] directory -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.atlassian.maven.plugins:maven-clover2-plugin:3.3.1:setup (clover-instrument) on project apex-lang: Clover has failed to instrument the source files in the [/home/bbusjaeger/dev/app/main/core/apex-lang/target/clover/src-instrumented] directory
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108)
at io.takari.maven.builder.smart.SmartBuilder$2.call(SmartBuilder.java:265)
at io.takari.maven.builder.smart.SmartBuilder$2.call(SmartBuilder.java:251)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.maven.plugin.MojoExecutionException: Clover has failed to instrument the source files in the [/home/bbusjaeger/dev/app/main/core/apex-lang/target/clover/src-instrumented] directory
at com.atlassian.maven.plugin.clover.internal.instrumentation.AbstractInstrumenter.instrumentSources(AbstractInstrumenter.java:198)
at com.atlassian.maven.plugin.clover.internal.instrumentation.AbstractInstrumenter.instrument(AbstractInstrumenter.java:71)
at com.atlassian.maven.plugin.clover.CloverInstrumentInternalMojo.execute(CloverInstrumentInternalMojo.java:287)
at com.atlassian.maven.plugin.clover.CloverSetupMojo.execute(CloverSetupMojo.java:30)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 11 more

Could you paste the exact error message?

I tried the published snapshot (I renamed it to release versions to hook it into our build). I got errors of the form "could not instrument source files under 'target/src-instrumented' for various projects.

Hi Benjamin,

I've changed a logic in the AbstractInstrumenter.redirectSourceDirectories(), it seems to work in various configurations (java only, groovy only, java + groovy; with gmaven plugin; with groovy-eclipse-plugin; with maven build-helper-plugin).

However, I would be very grateful if you could cross-check the fix in your project. Please use a 3.3.1-SNAPSHOT build of maven-clover2-plugin. It's available on the maven.atlassian.com:

• https://maven.atlassian.com/content/repositories/atlassian-public-snapshot/com/atlassian/maven/plugins/maven-clover2-plugin/3.3.1-SNAPSHOT/

Cheers
Marek

Now it's clear for me. Thanks a lot!

Yes, the build helper is called first. The clover plugin also picks up the extra source directory, i.e. it instruments all the source files in it. The problem is just that it still keeps java/src-build as an additional compile source root on the maven project (as shown in the code above).

Could you tell which one is called first: a build-helper adding extra source root or the clover2:setup?
I'm asking, because extra source roots should usually be added before the clover2:setup.

I have read those articles and debugged the maven plugin code. I am pretty sure it's a bug in this code:

        final CloverSourceScanner scanner = getSourceScanner();
        for (final String sourceRoot : sourceRoots) {
            if (new File(oldSourceDirectory).exists() && sourceRoot.equals(oldSourceDirectory)) {
                // if compilation root is the same as original source directory:
                // a) if it's a Java directory then use location of instrumented sources instead of the original source
                // root (e.g. 'src/main/java' -> 'target/clover/src-instrumented')
                // b) if it's a Groovy directory then don't change the location because we don't instrument Groovy on
                // a source level, so the Clover's instrumented folder is empty; Groovy files will be instrumented
                // during compilation on the AST level (e.g. 'src/main/groovy' -> 'src/main/groovy')
                if (scanner.isSourceRootForLanguage(sourceRoot, Language.Builtin.GROOVY))  {
                    addCompileSourceRoot(sourceRoot);
                } else {
                    addCompileSourceRoot(getSourceDirectory());
                }
            } else if ( !(getConfiguration().includesAllSourceRoots() && isGeneratedSourcesDirectory(sourceRoot)) ) {
                // if includeAllSourceRoots=true then ignore the original generated sources directory (e.g. target/generated/xyz),
                // because Clover will instrument them and store instrumented version (e.g. target/clover/src-instrumented);
                // compiler should know only the latter location, otherwise we would end up with the same classes included twice
                // (one with and one without Clover instrumentation)
                addCompileSourceRoot(sourceRoot);
            }
        }

This check fails for 'java/src-build', so it's added as a source dir: isGeneratedSourcesDirectory(sourceRoot)

Hi Benjamin,

Could you have a look at the following articles, please:

• https://confluence.atlassian.com/display/CLOVERKB/Duplicate+class+errors+with+Clover+and+JAXB+or+JAXB2+plugin
• https://confluence.atlassian.com/display/CLOVERKB/Duplicate+class+errors+with+Clover+and+jaxws-maven+plugin

They may be helpful.

Cheers
Marek