Unit and Integration tests coverage with Jacoco & SonarQube for multi product flavor builds

Kliment Joncheski
Sep 2, 2018 · 7 min read

Nowadays more and more developers tend to do TDD(although in 90% of the cases tests are written after the actual implementation of the feature). Nevertheless, what makes me happy is that is that developers tend to write more tests and tend to cover as much as they can from their codebase — even in the mobile world :). That is why this article can help you to setup two powerful tools in you Android project for code coverage. What is cool about it, is that reports for unit tests and Integration tests can be merged, resulting in clear reports for test coverage. Lets start!

First, lets get our hands on Jacoco. Jacoco in the nutshell works only with compiled java classes. Works on the principle of instrumentation (instrumentation is when source is changed — in this case the compiled .class files are modified on the fly, and Jacoco make another binary data
called .exec (executable). With this combination all the needed analyzis are made. Nothing more is needed)
But in order to be able to see nice & fancy report with the percentage with UI and clicking etc. the source classes (.java) are needed. More info about Jacoco can be found here: https://www.jacoco.org/jacoco/trunk/doc/.

So I’ve created simple android app. (source can be found here: https://github.com/kliment-jonceski/JacocoCoverageExampleApp) and added unit & ui (espresso) tests. First lets check the tasks needed to execute unit & ui tests. Within the project I’ve defined two flavors: “paid” and “free”, so here are the generated gradle tasks for executing unit & ui tests:

Lets start from the bottom. Unit tests execution tasks:

testFreeDebugUnitTest” = Run unit tests for the freeDebug build.

testFreeReleaseUnitTest” = Run unit tests for the freeRelease build.

testPaidDebugUnitTest” = Run unit tests for the paidDebug build.

testPaidReleaseUnitTest” = Run unit tests for the paidRelease build.

UI test execution tasks:

connectedFreeDebugAndroidTest” = Installs and runs the tests for freeDebug on connected devices.

connectedPaidDebugAndroidTest” = Installs and runs the tests for paidDebug on connected devices.

So it’s time to generate report with Jacoco. Lets create another jacoco-config.gralde file that will hold the whole Jacoco configuration.

/**
* Apply it in the module build.script.
*/

apply plugin: 'jacoco'

project.afterEvaluate {
def flavors = android.productFlavors.collect { flavor -> flavor.name }

// If there are no flavors defined, add empty flavor.
if (!flavors) flavors.add('')

def buildType = "Debug"
flavors.each { productFlavorName ->
def taskNameWithFlavor, pathNameWithFlavor
if (!productFlavorName) {
//empty flavors
taskNameWithFlavor = pathNameWithFlavor = "${buildType}"
} else {
taskNameWithFlavor = "${productFlavorName}${buildType}"
pathNameWithFlavor = "${productFlavorName}/${buildType}"
}
def unitTestsTaskName = "test${taskNameWithFlavor.capitalize()}UnitTest"
def
uiTestsTaskName = "connected${taskNameWithFlavor.capitalize()}AndroidTest"

// Create coverage task ex: 'jacocoTestReport<Flavor>' depending on
// 'testFlavorDebugUnitTest - unit tests' & connectedFlavorDebugAndroidTest - integration tests.
task "jacocoTestReport${productFlavorName.capitalize()}"(type: JacocoReport, dependsOn: [unitTestsTaskName, uiTestsTaskName]) {
group = "Reporting"
description = "Generate Jacoco coverage reports on the ${taskNameWithFlavor.capitalize()} build."

classDirectories = fileTree(
dir: "${project.buildDir}/intermediates/classes/${pathNameWithFlavor}",
excludes: ['**/R.class',
'**/R$*.class',
'**/BuildConfig.class',
'**/Manifest.class',
'**/Manifest$*.class',
'**/*$InjectAdapter.class',
'**/*$ModuleAdapter.class',
'**/*$ViewInjector*.class',
'**/*_MembersInjector.class',
'**/Dagger*Component.class',
'**/Dagger*Component$Builder.class',
'**/*Module_*Factory.class',
'**/*$ViewInjector*.*',
'**/*$ViewBinder*.*'
])

def coverageSourceDirs = [
"src/main/java",
"src/$productFlavorName/java",
"src/$buildType/java"
]
additionalSourceDirs = files(coverageSourceDirs)
sourceDirectories = files(coverageSourceDirs)
executionData = fileTree(dir: project.projectDir, includes: ["**/*.exec", "**/*.ec"])

reports {
xml.enabled = true
html.enabled = true
}
}
}
}

So lets see what we did there. Just apply the Jacoco plugin (no need to add dependency because Jacoco comes with the build tools for android). Then we create tasks for every flavor defined in our application for “debug” variant-> resulting in tasks: “jacocoTestReportFree” & “jacocoTestReportPaid” (if there are no flavors defined only one task will be created: “jacocoTestReport”). Important to notice here is that these created tasks are depending of the corresponding unit tests & integration tests gradle tasks:

jacocoTestReportFree depends on : testFreeDebugUnitTest, connectedFreeDebugAndroidTest

jacocoTestReportPaid depends on: testPaidDebugUnitTest, connectedPaidDebugAndroidTest

This means when we trigger jacocoTestReportFree task, the execution of testFreeDebugUnitTest, connectedFreeDebugAndroidTest tasks will take in place also-> resulting all unit & integration tests(ui tests) to be executed. Also note the classDirectories variable that defines the classes that will be analysed (we strip out classes that should not be included in the jacoco analysis like generated classses: R.class, dagger components etc). Important here is the executionData property where we include two files here. One is .exec = the resulting file from analysis of unit tests (after execution of testFreeDebugUnitTest). The other one is .ec file that will be generated after the analysis of integration tests (ui tests).

Important for integration tests to be executed is to enable integration tests in your build.gradle for the debug variant:

buildTypes {
debug {
testCoverageEnabled true
}
}

What is left from the configuration is to include this configuration within the app’s module biild.gradle script: apply from: ‘../jacoco-config.gradle’ . One last thing is that starting with version 3.0.0 of the Android plugin, now we cannot configure Jacoco using the android block DSL, but instead jacoco version must be defined in the classpath dependencies, alongside the Android plugin :

buildscript {
repositories {
google()
jcenter()
}
dependencies {
// Android plugin
classpath 'com.android.tools.build:gradle:3.0.1'
//Jacoco plugin
classpath 'org.jacoco:org.jacoco.core:0.8.1'
}
}

We are ready to go with Jacoco. Try to execute the desired task ex: jacocoTestReportFree, jacocoTestReportPaid (or just jacocoTestReport if no flavors are defined within the project). After execution of the task you should already see one report from jacoco: <project>/<app>/build/reports/jacoco/jacocoTestReport/html/index.html. If you try to open this file you should see report for unit tests generated by jacoco.

One hint that might save you some time: Run the jacoco analysis on emulator (executing this on Samsung devices won’t work).

Report from Jacoco when you open the index.html

Now lets setup the SonarQube. One part of this really powerful tool is the capability to show the test coverage & there are easy checks/reports that can monitor the code quality during development. More for SonarQube features you can find here.

If you haven’t setup Sonarqube yet, there are easy instructions here (you’ll need couple of minutes to install/setup SonarQube).

After the you’ve successfully configured SonarQube, we can define another sonarqube-config.gradle file in the project’s directory & add the whole SonarQube configuration there, similar as we did with the jacoco-config.gradle file.

apply plugin: 'org.sonarqube'//SonarQube configuration.
sonarqube {
def flavorName = ''
if (project.hasProperty('flavor')) {
//read the flavor from properties.
flavorName = flavor
androidVariant flavorName + "Debug"
} else {
androidVariant 'debug'
}
properties {
property 'sonar.host.url', 'http://localhost:9000'
property "sonar.projectName", "JacocoCoverageExampleApp"
property "sonar.projectKey", "jonceski.kliment.jacococoverageexample"
property 'sonar.java.source', '8'
property 'sonar.sourceEncoding', 'UTF-8'
property "sonar.language", "java"
property "sonar.login", "admin"
property "sonar.password", "admin"
property 'sonar.java.binaries', 'build/intermediates/classes/' + flavorName + "/debug"
property 'sonar.java.test.binaries', 'build/intermediates/classes/' + flavorName + "/debug"
property 'sonar.junit.reportsPath', 'build/test-results/test' + flavorName.capitalize() +'DebugUnitTest'
property 'sonar.jacoco.reportPath', fileTree(dir: project.projectDir, includes: ['**/*.exec'])
property 'sonar.jacoco.itReportPath', fileTree(dir: project.projectDir, includes: ['**/*.ec'])
property 'sonar.java.coveragePlugin', 'jacoco'
property 'sonar.exclusions', '**/*.js,**/*.css,**/*.html'
}
}

So lets shortly see what we’ve added there. Firstly the SonarQube plugin must be applied. Afterwards there are some properties that should be defined so our project can be created on sonar. First part of the properties are self-explanatory. One important property here is the projectKey -> which should be different for all projects created on SonarQube. Second part of the configuration is more interesting. Properties that are must: sonar.java.binaries (define where the sourceSets of binary files is), sonar.jacoco.reportPath (define where the jacoco report is generated for unit tests -> *.exec file), sonar.jacoco.itReportPath (define where the jacoco report is generated for integration tests -> *.ec file). Others properties are for excluding some files from the analysis (.css, .js files), or some additional informations like the total number of unitTests etc.

Lastly there one more part from the script that needs to be explained. The first lines after opening the SonarQube tag for configuration:

def flavorName = ''
if (project.hasProperty('flavor')) {
//read the flavor from properties.
flavorName = flavor
androidVariant flavorName + "Debug"
} else {
androidVariant 'debug'
}

With this code block we read the defined project property flavor via the execution command that will be executed. If that property is not set we define empty flavor. Similar to what we did before, apply the sonarqube-config.gradle in the app/build.gradle file. Note the sonarqube also the version of SonarQube should be defined within the classpath dependencies:

buildscript {
repositories {
google()
jcenter()
}
dependencies {
// Android plugin
classpath 'com.android.tools.build:gradle:3.0.1'
//Jacoco plugin
classpath 'org.jacoco:org.jacoco.core:0.8.1'
//SonarQube plugin
classpath 'org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.6.2'
}
}

Finally the command for free/paid product flavors would be:

./gradlew clean jacocoTestReportFree sonarqube -Pflavor=free

./gradlew clean jacocoTestReportPaid sonarqube -Pflavor=paid

If there is no flavor define simply call:

./gradlew clean jacocoTestReport sonarqube

After the execution there should be new project created on SonarQube & there should be the reports for test coverage (both unit tests & integration tests). Check the following screenshots.

Sonarqube overview test coverage.
Detailed & easy to read report for test coverage generated by Sonarqube.

Full code for Jacoco& SonarQube added here. Happy testing.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade