Android UI and unit tests coverage report with Jacoco and SonarQube

Nowadays writing code without tests is a sign of poor tone. So everybody in our team is trying to improve one’s skills in this field. Writing tests became a competition for us (approaches, libs etc.). But some time ago we faced a problem — how to get full tests coverage of our android app code!?!?! We found out, that there are no problems to get pure unit test coverage or androidTest coverage but when we wanted to get FULL (merged) coverage we were surprised — it is not quit trivial task!

So here we`d like to show our example of how we`ve merged coverage and presented it in awesome and readable way.

To give you an example we`ve developed an easy app, its goal is to send a request to remote server which returns an IP of client and show this IP in TextView. The result is something like this (Pic. 1):

Picture 1

To make our life easier while testing we`ve chosen MVP pattern and DI with Dagger2 for this app. You can find package structure in Pic.2

Picture 2

Rest API we implemented with RX + Retrofit2 (such a surprise) injecting ServerAPI so that substitute it latter in UI tests:

@Module
public class ServerModule {
private static final String BASE_URL = “http://httpbin.org";
@Provides
@Singleton
public ServerApi provideServerApi(){
return getServerApi();
}
protected ServerApi getServerApi() {
    Retrofit mRetrofit = new Retrofit.Builder()
        .addCallAdapterFactory(RxJavaCallAdapterFactory.create())
        .addConverterFactory(GsonConverterFactory.create())
        .baseUrl(BASE_URL)
        .build();
    return mRetrofit.create(ServerApi.class);
  }
}

So when User clicks “GET IP” button our activity triggers its presenter request method:

public class MainActivity extends AppCompatActivity implements MainActivityContract {
private MainActivityPresenter mPresenter;
@Inject
protected ServerApi mServerApi;
private ActivityMainBinding mBinding;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mBinding = DataBindingUtil.setContentView(this, R.layout.activity_main);
((CoverageApp) CoverageApp.provideAppContext()).dataComponent().inject(this);
mPresenter = new MainActivityPresenter(mServerApi, this);
mBinding.getIpBtn.setOnClickListener(v -> mPresenter.getIp());
}
@Override
public void onSuccess(String origin) {
mBinding.ipTv.setText(origin);
}
@Override
public void onError(String error) {
mBinding.ipTv.setText(error);
}
}

And when presenter is ready it triggers one of two callback methods:

public class MainActivityPresenter {
private ServerApi mServerApi;
private MainActivityContract mCallBack;
public MainActivityPresenter(ServerApi serverApi, MainActivityContract callBack) {
this.mCallBack = callBack;
this.mServerApi = serverApi;
}
public void getIp() {
mServerApi.getIp()
.subscribeOn(AppSchedulers.io())
.observeOn(AppSchedulers.mainThread())
.subscribe(ResponseModel -> {
if (null != ResponseModel.getOrigin() && !ResponseModel.getOrigin().isEmpty()) {
mCallBack.onSuccess(ResponseModel.getOrigin());
} else {
mCallBack.onError(“Error”);
}
}, error -> mCallBack.onError(error.getMessage()));
}
}

Here one may notice that we used custom Schedulers for RX. Its goal is to convert async calls of RX into immediate ones in our tests. Everything else is quite obvious.

Let`s go down to our test.

We`ve made three tests (just for example):

· Unit test

· UI Espresso test

· Robotolectric test — this one appeared to be quite tricky so we decided to show it separately

Let`s start from the unit test

Here we used Mockito for mocks and that`s it:

public class MainActivityPresenterTests {
private MainActivityPresenter mPresenter;
@Rule
public SynchronousSchedulers schedulers = new SynchronousSchedulers();
@Mock
private MainActivityContract mCallBack;
@Mock
private ServerApi mApi;
@Before
public void setUp() {
MockitoAnnotations.initMocks(this);
}
@Test
public void test_goodResponse_getIp() throws Exception {
ResponseModel response = new ResponseModel();
String IP = “192.168.0.1”;
response.setOrigin(IP);
when(mApi.getIp()).thenReturn(Single.just(response));
mPresenter = new MainActivityPresenter(mApi, mCallBack);
mPresenter.getIp();
verify(mCallBack, times(1)).onSuccess(anyString());
}
@Test
public void test_error_getIp() throws Exception {
when(mApi.getIp()).thenReturn(Single.error(new ConnectException(“Error”)));
mPresenter = new MainActivityPresenter(mApi, mCallBack);
mPresenter.getIp();
verify(mCallBack, times(1)).onError(“Error”);
}
}

UI Espresso test:

@RunWith(AndroidJUnit4.class)
public class MainActivityTest {
private MockServer mMockServer;
private final MyTestRule component =
new MyTestRule(InstrumentationRegistry.getTargetContext());
private final ActivityTestRule<MainActivity> activityRule =
new ActivityTestRule<>(MainActivity.class, false, false);
@Rule
public TestRule chain = RuleChain.outerRule(component).around(activityRule);
@Before
public void setUp(){
CoverageApp mApp = (CoverageApp) getInstrumentation().getTargetContext().getApplicationContext();
mMockServer = (MockServer) mApp.dataComponent().getServerApi();
}
public void launchActivity(){
activityRule.launchActivity(null);
}
@Test
public void test_goodResponse(){
ResponseModel response = new ResponseModel();
String IP = “192.168.0.1”;
response.setOrigin(IP);
mMockServer.setResponse(response);
launchActivity();
onView(withId(R.id.get_ip_btn)).perform(click());
onView(withId(R.id.ip_tv)).check(matches(withText(IP)));
}
}

Here with custom rule we substituted dataComponent to inject MockServerApi then we can put response data we want to and check UI behavior

And at last Robotolectric test. We`ve written it as “test for test” only with one aim — include robotolectric test in app. It tests nothing )))

@RunWith(RobolectricTestRunner.class)
@Config(constants = BuildConfig.class, sdk = 21)
public class MainActivityPresenterRobo {
private MainActivity mActivity;
private ActivityController<MainActivity> actControl;
@Before
public void setUp(){
actControl = Robolectric.buildActivity(MainActivity.class);
mActivity = actControl.create().get();
}
@Test
public void test_click() throws Exception{
actControl.resume();
TextView ipTv = (TextView) mActivity.findViewById(R.id.ip_tv);
assertEquals(“Here will appear your IP”, ipTv.getText());
}
}

Ok, now having these tests we can go to coverage and measure it. As it was written above there is no obvious way to get merged test coverage and only thing we can do is get separate coverage reports.

Get unit test coverage:

Go to your unit tests and run tests with coverage Pic. 3

Picture 3

After all your tests passed you will see results like in pictures 4 and 5

Picture 4

Picture 5

Now let`s run androidTests coverage. For this you`ll have to set up your gradle file in an appropriate way (we`ll discuss it latter) then go to gradle bookmark in your Android Studio, find there task “createDebugCoverageTest” and now you can find results here: “YOUR_PROJECT_PATH\app\build\reports\coverage\debug\index.html”.

Open it and find nice pictures of your UI test coverage (pic 6)

Picture 6

All this is fine but what about merging these results?

We`ve already got unit, android and robotolectric tests and we`d like to get merged coverage results for them to decide whether our project meets company requirements or not, also we`ll analyze the code style.

All of this we are going to do with Jacoco and SonarQube. We`ll set up both of them in the separate *.gradle files (jacoco.gradle and sonarqube.gradle) and apply them in our build.gradle file like this:

apply from: ‘./fileName.gradle’

Also pay attention that one must put the following lines in app/build.gradle to enable coverage calculation:

android{
debug {
testCoverageEnabled true
}
}

Now let`s consider Jacoco:

Jacoco — is a free tool for java project tests coverage calculation. Jacoco is actively supported and updates are released quite often. You can find details here Jacoco.

At the beginning here is listing of the jacoco task:

apply plugin: ‘jacoco’
jacoco {
toolVersion “0.7.6.201602180812”
}
// run ./gradlew clean createDebugCoverageReport jacocoTestReport
task jacocoTestReport(type: JacocoReport, dependsOn: “testDebugUnitTest”) {
group = “Reporting”
description = “Generate Jacoco coverage reports”
reports {
xml.enabled = true
html.enabled = true
}
def fileFilter = [‘**/R.class’,
‘**/R$*.class’,
‘**/BuildConfig.*’,
‘**/Manifest*.*’,
‘android/**/*.*’,
‘**/Lambda$*.class’, //Retrolambda
‘**/Lambda.class’,
‘**/*Lambda.class’,
‘**/*Lambda*.class’,
‘**/*Lambda*.*’,
‘**/*Builder.*’,
‘**/*_MembersInjector.class’, //Dagger2 generated code
‘**/*_MembersInjector*.*’, //Dagger2 generated code
‘**/*_*Factory*.*’, //Dagger2 generated code
‘**/*Component*.*’, //Dagger2 generated code
‘**/*Module*.*’ //Dagger2 generated code
]
def debugTree = fileTree(dir: “${buildDir}/intermediates/classes/debug”, excludes: fileFilter)
def mainSrc = “${project.projectDir}/src/main/java”
sourceDirectories = files([mainSrc])
classDirectories = files([debugTree])
executionData = fileTree(dir: project.projectDir, includes:
[‘**/*.exec’ , ‘**/*.ec’])
}

Now let`s discuss what is going on inside:

  1. Add jacoco plugin for gradle
  2. Set its version
  3. Then goes task which will trigger the tests and merge their results. You can find more info about gradle tasks here

Long story in short: we write taskName, set its type and tasks which it depends on, its group and description. Tell it what kind of reports it must generate.

Then we create fileFilter — list of files which must be excluded from the coverage analysis. In our case they are generated android files, retrolambdas and Dagger files.

Set sourceDirectories — where our files are. Set classDirectories — class files, do not forget to add earlier created fileFilter into exclude.

And now the most interesting part — we create executionData — from where task will get merge info.

Unit tests coverage results are stored into *.exec file (YOUR_PROJECT_PATH\app\build\jacoco\*.exec),

androidTests in *.ec file

(YOUR_PROJECT_PATH\app\build\outputs\code-coverage\connected\*.ec)

Well, it seems to be enough for the moment and let`s now try it:

open console and run following sequence of tasks:

./gradlew clean createDebugCoverageReport jacocoTestReport

Sequence is IMPORTANT: androidTests then unit tests (our task triggers “testDebugUnitTests”)

And now we can enjoy the results — go to YOUR_PROJECT_PATH\app\build\reports\jacoco\jacocoTestReport\html

and find there coverage results.

Everything seems ok and few moments we were happy, but then we discovered little feature!

Do you remember we told you about Robotolectric tests? The feature is that they are not included in this report :(

To include them you have to add to your app/build.gradle these lines:

android{
testOptions {
unitTests.all {
jacoco {
includeNoLocationClasses = true
}
}
}
}

And that`s it. Rerun your task and happiness comes.

At this point one could stop, but we went on and decided to feed our coverage report to sonarQube so that to have all code quality factors in one basket.

SonarQube

You can find SonarQube here

Short story: download sonarQube, install it, start sonarQube server. If something went wrong (but I really doubt it can happen) try google, there are lots of info about sonarQube.

You can add lots of plugins for detailed analyze of your code. In our case we added following plugins: Android, CheckStyle, FindBugs, Git, Java, XML. Also you can create customRules up to your taste and requirements (more here).

As soon as you managed your sonarQube return to android project and consider sonarQube task:

apply plugin: ‘org.sonarqube’
ext {
SONAR_HOST = “http://localhosts:9000/"
}
sonarqube() {
properties {
/* SonarQube needs to be informed about your libraries and the android.jar to understand that methods like
* onResume() is called by the Android framework. Without that information SonarQube will very likely create warnings
* that those methods are never used and they should be removed. Same applies for libraries where parent classes
* are required to understand how a class works and is used. */
def libraries = project.android.sdkDirectory.getPath() + “/platforms/android-24/android.jar,” +
“${project.buildDir}/intermediates/exploded-aar/**/classes.jar”
property “sonar.projectName”, (String) android.defaultConfig.applicationId
property “sonar.projectKey”, android.defaultConfig.applicationId + android.defaultConfig.versionName
property “sonar.sourceEncoding”, “UTF-8”
property “sonar.sources”, “./src/main/”
property “sonar.libraries”, libraries
property “sonar.binaries”, “/intermediates/classes/debug”
property “sonar.java.binaries”, “${project.buildDir}/intermediates/classes/debug”
property “sonar.java.libraries”, libraries
property “sonar.exclusions”, “build/**,**/*.png,*.iml, **/*generated*, “
property “sonar.import_unknown_files”, true
property “sonar.android.lint.report”, “./build/outputs/lint-results.xml”
property “sonar.host.url”, SONAR_HOST
property “sonar.tests”, “./src/test/, ./src/androidTest/”
property “sonar.jacoco.reportPath”, fileTree(dir: project.projectDir, includes: [‘**/*.exec’])
property “sonar.java.test.binaries”, “${project.buildDir}/intermediates/classes/debug”
property “sonar.jacoco.itReportPath”, fileTree(dir: project.projectDir, includes: [‘**/*.ec’])
property “sonar.java.test.libraries”, libraries
}
}

Description:

1. Apply plugin for gradle

2. Put sonarQube host in extensions for better readability

3. Describe sonarQube properties. More about properties and examples.

In our case put attention on these properties:

sonar.java.test.binaries
sonar.java.binaries
sonar.tests
sonar.jacoco.reportPath
sonar.jacoco.itReportPath

Ok, let`s start it:

./gradlew clean createDebugCoverageReport jacocoTestReport sonarqube

Now go to sonarQubeHost and find your awesome results:

Or like this

Also you`ll find a lot of various info about your tests going deeper in this report, for instance condition coverage, line coverage etc.

Do not forget that sonarQube is quite powerful static code analyzer — inspect your code with it and make your project better.

So “May the force be with you”. Do not hesitate to contact us.

Igor Torba — https://www.facebook.com/TorbaIgor

Sergiy Grechukha — https://www.facebook.com/sergiy.grechukha