Open in app

Sign in

Write

Sign in

Android+Kotlin Code coverage with Jacoco, Sonar and Gradle plugin 6.x

Vanda Cabanova
Jamf Engineering
Published in
7 min readSep 21, 2020

--

This blog post was written under the Wandera brand prior to its acquisition by Jamf. That’s history, we are now #OneJamf

At Wandera, we have been measuring our code coverage for quite a while, but we relied on a 3rd party plugin that solved the Jacoco plugin configuration for us because we didn’t want to deal with Groovy language for speed and simplicity.

We were dealing with two issues — we were using an outdated Gradle plugin and we were missing the instrumentation test coverage in our reports.

When we upgraded the Gradle plugin to the newest version (6.6.1 as of writing this blogpost), it wasn’t all sunshine and rainbows with the then-used 3rd party plugin as it had not been maintained for a good year.

I learned the plugins are quite simple to implement, so to encourage you not to use 3rd party plugins when it is not necessary, I’d like to present you with a thorough guideline.

We are going to go through how to setup Jacoco plugin, how to setup the client part of Sonar, how to be able to choose one or both the jvm and instrumentation tests code coverage and what to watch out for when using version 6.x of the Gradle plugin.

Jacoco plugin

Firstly, you will want to add Jacoco plugin to your project.
Add classpath “org.jacoco:org.jacoco.core:0.8.5” to your project.

buildscript {
    repositories { 
        google()
        jcenter()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:4.0.1'
        ...
        classpath "org.jacoco:org.jacoco.core:0.8.5"
    }
}

—
Version 0.8.5 contains a small bug so if you want to avoid any issues, use a different version instead
https://github.com/jacoco/jacoco/issues/968
—

Coverage configuration

Now we will want to compute coverage of JVM unit tests and Android instrumentation tests.

I got inspired mostly by reading https://medium.com/@azizbekian/setup-jacoco-sonarqube-in-multimodule-multiflavor-kotlin-android-project-d8e7b27aed36 so kudos to you, @azizbekian!

You can configure Jacoco in your module’s build.gradle file, but it is much cleaner to have larger configurations separated in new gradle files for easier maintenance. A separate file may also be reused across modules or projects.

All Jacoco properties are explained at https://docs.gradle.org/current/dsl/org.gradle.testing.jacoco.tasks.JacocoReport.html

Contents of the jacoco.gradle file (located where the module’s build.gradle file is) explained below:

apply plugin: 'jacoco'

jacoco {
    toolVersion = "0.8.5"
    // Custom reports directory can be specfied like this:
    // reportsDir = file("$buildDir/customJacocoReportDir")
}

tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
    jacoco.excludes = ['jdk.internal.*']
    // see related issue https://github.com/gradle/gradle/issues/5184#issuecomment-457865951
}

project.afterEvaluate {

    (android.hasProperty('applicationVariants')
            ? android.'applicationVariants'
            : android.'libraryVariants')
            .all { variant ->        def variantName = variant.name
        def unitTestTask = "test${variantName.capitalize()}UnitTest"
        def androidTestCoverageTask = "create${variantName.capitalize()}CoverageReport"

        tasks.create(name: "${unitTestTask}Coverage", type: JacocoReport, dependsOn: [
                "$unitTestTask",
                "$androidTestCoverageTask"
        ]) {
            group = "Reporting"
            description = "Generate Jacoco coverage reports for the ${variantName.capitalize()} build"

            reports {
                html.enabled = true
                xml.enabled = true
                csv.enabled = true
            }

            def excludes = [
                  // data binding
                  'android/databinding/**/*.class',
                  '**/android/databinding/*Binding.class',
                  '**/android/databinding/*',
                  '**/androidx/databinding/*',
                  '**/BR.*',
                  // android
                  '**/R.class',
                  '**/R$*.class',
                  '**/BuildConfig.*',
                  '**/Manifest*.*',
                  '**/*Test*.*',
                  'android/**/*.*',
                  // butterKnife
                  '**/*$ViewInjector*.*',
                  '**/*$ViewBinder*.*',
                  // dagger
                  '**/*_MembersInjector.class',
                  '**/Dagger*Component.class',
                  '**/Dagger*Component$Builder.class',
                  '**/*Module_*Factory.class',
                  '**/di/module/*',
                  '**/*_Factory*.*',
                  '**/*Module*.*',
                  '**/*Dagger*.*',
                  '**/*Hilt*.*',
                  // kotlin
                  '**/*MapperImpl*.*',
                  '**/*$ViewInjector*.*',
                  '**/*$ViewBinder*.*',
                  '**/BuildConfig.*',
                  '**/*Component*.*',
                  '**/*BR*.*',
                  '**/Manifest*.*',
                  '**/*$Lambda$*.*',
                  '**/*Companion*.*',
                  '**/*Module*.*',
                  '**/*Dagger*.*',
                  '**/*Hilt*.*',
                  '**/*MembersInjector*.*',
                  '**/*_MembersInjector.class',
                  '**/*_Factory*.*',
                  '**/*_Provide*Factory*.*',
                  '**/*Extensions*.*',
                  // sealed and data classes
                  '**/*$Result.*', 
                  '**/*$Result$*.*'
            ]

            def javaClasses = fileTree(dir: variant.javaCompileProvider.get().destinationDir, 
excludes: excludes)            def kotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlin-classes/${variantName}", 
excludes: excludes)

            classDirectories.setFrom(files([
                    javaClasses,
                    kotlinClasses
            ]))

            def variantSourceSets = variant.sourceSets.java.srcDirs.collect { it.path }.flatten()
            sourceDirectories.setFrom(project.files(variantSourceSets))

            def androidTestsData = fileTree(dir: "${buildDir}/outputs/code_coverage/${variantName}AndroidTest/connected/", includes: ["**/*.ec"])

            executionData(files([
                    "$project.buildDir/jacoco/${unitTestTask}.exec",
                    androidTestsData
            ]))
        }

    }


}

Gradle tasks

We created a task for each variant of our app. So if you have a debug, dev and release variant, you should see testDebugUnitTestCoverage, testDevUnitTestCoverage and testReleaseUnitTestCoverage Gradle tasks in your right-side Gradle tab in Android Studio. You may name the jobs whatever you like.

Then we need to declare that our testXxxUnitTestCoverage task depends on running the tests so that we have some data to compute the coverage from. We cannot start with computation until the necessary output files are generated.

To enable Android Instrumentation tests coverage, your module’s build.gradle file should look like this:

buildTypes {      
    debug {         
      applicationIdSuffix ".debug"         
      minifyEnabled false
      testCoverageEnabled true
    }
}

Important: the Android Instrumentation test coverage works only if you set your application variant’s property testCoverageEnabled true

When the test coverage is not enabled, the testXxxUnitTestCoverage tasks for such variants won’t work unless you exclude androidTestCoverageData from dependencies (for instance by not depending on the task and by not expecting the report to be available for execution data).

On the other hand, you only need to keep track of one of your variants which usually is the debug one.

Report files

Moving on, we specify the types of reports we want to have generated.

reports {
    html.enabled = true
    xml.enabled = true
    csv.enabled = true
}

HTML format is the most human-friendly. XML and CSV reports are used for other reporting tools, for instance Sonar or a CI plugin.

Sources

We need to tell Jacoco where to find our Java and Kotlin classes

def javaClasses = fileTree(dir: variant.javaCompileProvider.get().destinationDir, 
excludes: excludes)def kotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlinclasses/${variantName}", 
excludes: excludes)

You can see the build directory can be referenced variably.

If you don’t include the java classes, any tests written in java won’t be taken into account. Same for Kotlin.

Then we want to exclude generated classes that we cannot cover with tests and would make the coverage unnecessarily low. These are usually classes generated by 3rd parties during compilation, such as Dagger/Hilt, Butterknife, Kotlin etc. You may exclude anything you consider unnecessary to appear in your reports.

I enriched my list by looking in the following sources.

Gradle plugin 6.x update

In one of the newest Gradle plugin versions, some properties were made read-only. That’s one of the reasons most of the abandoned 3rd party plugins have stopped working. Setter methods must be used instead.

So instead of a direct assignment, setters must be used.

classDirectories.setFrom(files([
                    javaClasses,
                    kotlinClasses
            ]))

You may want to add certain directories for a specific variant type.

You can do so in

def variantSourceSets = 
variant.sourceSets.java.srcDirs.collect { it.path }.flatten()

Tests data

At last we need to specify where to find the execution data.

def unitTestsData = "$project.buildDir/jacoco/${unitTestTask}.exec"def androidTestsData = fileTree(dir: "${buildDir}/outputs/code_coverage/${variantName}AndroidTest/connected/", 
includes: ["**/*.ec"])

executionData(files([unitTestsData, androidTestsData]))

An .exec file is created for the JVM unit tests and a similar file of .ec type is created for Android instrumentation files.

We define both of these paths using the Jacoco’s executionData property.

If you only want to track coverage of your unit tests, don’t add the androidTestsData path. The same goes for the instrumentation tests.

For future reference, some Android plugin versions change the build dir hierarchy, so in case it is not working in the future, make sure the files are generated at the path you provided. If not, change the path to the directory the files now appear in.

You can find the description of each Jacoco property by following the below link.

https://docs.gradle.org/current/dsl/org.gradle.testing.jacoco.tasks.JacocoReport.html#org.gradle.testing.jacoco.tasks.JacocoReport:executionData

Apply the plugin

Now that we have the plugin configuration done, we need to make sure it is applied.

In your app’s build.gradle file, add the following line apply from: “jacoco.gradle” so it looks like the following:

apply plugin: 'com.android.application'
...
apply from: "jacoco.gradle"

Now when you run

./gradlew testDebugUnitTestCoverage

you should see your reports appear in the Jacoco directory.

Watch out for the Gradle plugin version!

This whole story started with updating the Android Gradle plugin to version 4 as the Android Studio likes to warn us about new updates.

When you update the Android Gradle plugin version, it usually comes with updating the Gradle wrapper version to be compatible as per the documentation on the official site https://developer.android.com/studio/releases/gradle-plugin

Gradle plugin in your project's gradle.build file

buildscript {

    repositories {
        google()
        jcenter()
    }
    
    dependencies {
        classpath 'com.android.tools.build:gradle:4.0.1'
        ...
    }
}

And when you are updating, you usually select the latest version possible. So we did update the gradle-wrapper.properties file:

distributionUrl=https\://services.gradle.org/distributions/gradle-6.6.1-all.zip

After that, our automated builds started failing on not having the Jacoco reports available!

After the whole day and night of searching for all possible issues with Jacoco I finally bumped into the Gradle issues Github site saying the Gradle plugin version 6.4 makes the Android connected tests run separately and that Jacoco does not wait for them to finish to pick up the report.

https://github.com/gradle/gradle/issues/14132 — Connect to preview

All you need to do though is to downgrade to 6.3 and wait for a fix

distributionUrl=https\://services.gradle.org/distributions/gradle-6.3-all.zip

Configure Sonar

You want to continue reading if you have your Jacoco reports ready and you want to be able to automate the visualisation and you would like to use the static analysis that Sonar offers.

This section assumes you know how to setup the server side and you know a little about Sonar.

You may again configure sonar in a separate gradle file, such as sonar.gradle and put it next to jacoco.gradle and not to forget, add “apply from” to the main build.gradle file.

Add a new dependency in your module’s gradle file

buildscript {
    repositories {
        jcenter()
        google()
    }    dependencies {
        classpath "org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.7.1"
        ...
    }
}

We still use the 2.7.1 version, but I think it is safe to update to 2.8.

Then add this at the bottom (or in your newly created sonar.gradle file)

sonarqube {
    properties {        property "sonar.host.url", "https://sonar.yourhostname.net"
        property "sonar.projectName", "Android"
        property "sonar.projectKey", "your.project.group:module"
        property "sonar.projectVersion", "${android.defaultConfig.versionName}"
        
        property "sonar.junit.reportsPath", "build/test-results/testDebugUnitTest"property "sonar.java.coveragePlugin", "jacoco"
        property "sonar.sourceEncoding", "UTF-8"
        property "sonar.android.lint.report", "build/reports/lint-results.xml"property "sonar.jacoco.reportPaths", "build/jacoco/testDebugUnitTest.exec"
        property "sonar.jacoco.itReportPath", fileTree(dir: project.projectDir, includes: ["**/*.ec"])
        property "sonar.coverage.jacoco.xmlReportPaths", "build/reports/jacoco/testDebugUnitTestCoverage/testDebugUnitTestCoverage.xml"
    }
}tasks.sonarqube.dependsOn ":app:testDebugUnitTestCoverage"

The properties are described here https://docs.sonarqube.org/latest/analysis/analysis-parameters/

The pasted code snippet should work for all Sonar plugin versions, although some of the parameters may be deprecated.

If you want to have the instrumentation tests coverage included, make sure you add the “sonar.jacoco.itReportPath” property.

Android
Sonar
Jacoco
Code Coverage
Gradle

--

--

Vanda Cabanova
Jamf Engineering

Written by Vanda Cabanova

14 Followers

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams