Improving Code Coverage In Android App
Writing mobile apps is hard. Writing good and maintainable apps is even harder. During development we want to make to sure, that every change in codebase won’t degrade quality and functionality.
It’s hard to imagine nowadays that you can make a solid and maintainable application without writing tests and unit tests specifically. But we often face a question — how many tests are enough to make sure that this piece of code is properly covered by tests. Well, there is no clear answer, but today I want to look into a wonderful tool called Jacoco, which helps to keep your valuable pieces of code covered by unit tests.
Jacoco stands for Java Code Coverage tool and it’s been used by java developers for decades, but android developers can also use all the benefits of it if you configure it properly. There are a lot of articles in a wild of how to configure Jacoco for generating test coverage reports for android project, so I won’t go deep into that case. Instead I will show how to configure Jacoco automatic verification of test coverage so you will be able to introduce it to your build or CI pipeline.
Let’s take a look to a very simple app, which consists of two screens — login and main screen.
Please note, that this sample is made only for demo purposes only, and is not intended to be production ready
You can find a full project source on github
In this sample I implemented very basic MVP pattern. This means that we treat Activities as a passive view, and all valuable UI logic exist in Presenters. Let’s take a closer look at LoginPresenter. Here we have a keyboard input handling, login and password validation, and auth logic itself. If everything is ok, we are transitioning to MainActivity. There are some tests in LoginPresenterTest as well. The rule of thumb for MVP pattern is to have in View layer as low amount of code in View layer as possible, and thus, keep logic in presenters, which can easily be covered with unit tests. We will configure Jacoco in that manner. We will ignore View layer and some android specific classes, and will generate coverage report for the rest of classes.
If we have project structured by features, we are able to ignore all view classes in all future features by simply placing one filter pattern for classname
‘com/androidjacoco/sample/**/view/**.*’
Now, if we run
./gradlew customJacocoTestReportwe will have report something like that
Here we see that report shows coverage of some data classes and presenters.
We are interested in three columns:
Missed instructions — provides information about the amount of code that has been executed or missed and counts are single Java byte code instructions.
Missed Branches — this metric counts the total number of such branches in a method and determines the number of executed or missed branches.
Classes — a class is considered as executed when at least one of its methods has been executed.
More info about report details you can find here
Lets look at the class LoginPresenter and make sure that indeed we have a lot of missed instructions and branches.
Also, take a note, that anonymous classes, like SingleObserver or Consumer above, are counted as independent classes in report, as you can see in classes column for com.androidjacoco.sample.login.presenter package
Now, we don’t want code with such low test cover ratio to be passed to production, so what we can do about that? There is a JacocoCoverageVerification task to rescue!
This task depends on our previous task customJacocoTestReport, and if we run it, it will first generate coverage report and then will analyze it. It consists of some configuration for locating source classes and violationRules block. Block violationRules itself consists of a number of rule blocks, there can be as many such rule blocks, as you want. For this sample I made 3 rules:
- If we have missed instructions ratio more then 60%, then build will fail.
2. If we have 80% of branches(if, switches, etc) uncovered, then build will fail.
3. Here I declare, that all classes in presenter package MUST have at least one unit test. Please note that all anonymous classes also considered as independent classes.
More info about rules syntax you can read here
If we run
./gradlew customJacocoTestCoverageVerificationwe will get a rule violation error
FAILURE: Build failed with an exception.* What went wrong:
Execution failed for task ‘:app:customJacocoTestCoverageVerification’.> Rule violated for package com.androidjacoco.sample.login.presenter: branches covered ratio is 0.3, but expected minimum is 0.5
Rule violated for package com.androidjacoco.sample.login.presenter: instructions covered ratio is 0.1, but expected minimum is 0.8
Rule violated for package com.androidjacoco.sample.login.presenter: classes missed count is 3, but expected maximum is 0
We can see here, that our coverage ratio doesn’t satisfy our rules, hence we need to write more tests!
In the sample repository in branch VerificationPassed you can see the code, which coverage is satisfied with rules. In that manner, you can configure your build script, and include coverage verification into your build process.
Jacoco is a very powerful tool, which can help you to increase code coverage of your codebase. Of course, it’s up to you and your project requirements to define rules and ratio of covered code, but nevertheless it’s a good metrics of the codebase health, and it’s worth to try to introduce it to your build pipeline. In our project we started with ratio 0.4 and then it slowly increased to 0.6 and continues to grow.
