Merging Integration, Unit and functional test reports with JaCoCo

jaco and co :)

Code coverage is an important aspect in any software development. It tells you how much of your code was executed by your tests. As a tool, it is useful during the development of your unit/integration tests because it helps highlight where your tests may have missed paths through the code.

For example, if the code you are testing has an if..then..else condition and your test call the code with parameters that cause the if condition block to be tested but not the else block, code coverage helps highlight untested code.

There are many tools to generate code coverage reports. JaCoCo is a free code coverage library for Java, which has been created by the EclEmma team based on the lessons learned from using and integration existing libraries for many years.

Configuring jacoco in the pom.xml for maven projects to generate coverage reports for unit and integration is easy enough to get started with jacoco. But, there are certain times when you want a merged report that takes into account unit, integration, and functional tests reports together.

In this blog, we’ll go through the configuration options required to generate unit, integration, and functional test individual coverage reports. Since we are using different frameworks for testing like Junit and Cucumber / Karate Jacoco will also help to generate an overall coverage report that integrates the merged coverage data.

Setup

The multi-module maven project we are going to set up here comprises of 3 maven modules.

module1 and module2 comprise of one domain java class with some functions and has respective unit-integration test classes.

module3 comprises of cucumber functional test and also contains configuration for merging jacoco reports.

The snippet of pom modules and properties would look like below:

main pom.xml

We are also configuring few pom profiles which would help in activating the unit, integration-test and testAll profile.

profiles for unit and integration tests.

Configuring surefire plugin

The Surefire Plugin is used during the test phase of the build lifecycle to execute the unit tests of an application. We can exclude the test files which we don’t want surefire plugin to include in test phase using excludes property.

surefire plugin

Configuring failsafe plugin

The failsafe plugin is used for integration tests of a project. It has two goals:

1. integration-test — This goal is bound to the integration-test phase by default.
2. verify — verify that the integration tests passed. This goal is bound to the verify phase by default.

failsafe plugin

Including jacoco plugin

Jacoco splits collection of test coverage data from the generation of the coverage report. The below <execution> block tells jacoco to initiate agent for unit test and integration test.

jacoco agent initialize.

Now, we configure jacoco plugin in module3 for executing cucumber tests and there is an important note on argLine which should be noted and kept in mind. argLine helps in holding the command line params of test executions and later helps in generating merged reports.

Now we would configure jacoco maven plugin in module3 for merging and aggregating jacoco reports in different execution steps. Now we will add different goals in plugin.

report-aggregate : Creates a structured code coverage report (HTML, XML, and CSV) from multiple projects within reactor. The report is created from all modules this project depends on.

report-aggregate

merge: This goal helps in merging a set of execution data files (*.exec) into a single file.

Since every module has an individual *.exec file generated by jacoco. This merge goal merges those files specified in includes property into a single aggregate file to the given location in destFile.

merge reports.

We can also get a view of the goals report-aggregate and merge from the logs, which looks like below:

aggregate and merge logs.

Now to add an extra layer of additional condition, you can add a check goal in the plugin which would validate the rules given inside the execution block and would fail the build if the rules are n’t met. The dataFile property specifies the merged report location against which rules will get checked.

check execution.

This now ends the plugin configuration for jacoco. You may check the source code for exact list of dependencies used in project.

Now if you run mvn clean install -P testAll , this will activate testAll profile and generate the jacoco merged report.

Now you can view the merged report in browser by opening module3/target/site/jacoco-aggregate/index.html file.

index.html

Now we will try to ignore the module3 cucumber test and see the effect on jacoco report. You can achieve this by simply ignoring the tag specified in feature file. Run this command :mvn clean install -P testAll -Dcucumber.options="--tags ~@Concatenation"

ignored cucumber tests.

The result shows that function concatenation of module1 has not been invoked and thus coverage has been reduced and build failed.

Summary

We have seen now how to configure jacoco along with fail safe and sure fire plugins to get overall code coverage in maven projects. If you need additional configuration in jacoco and its rules visit the official docs. We have also tried to increase the coverage of module1 with module3 by using cucumber test framework. A similar approach would be followed if Karate API testing framework is used instead of cucumber. To learn more of karate integration with Spring boot visit my repo.

Source code is available here. Happy Learning !!