Screenshot testing in Espresso

This Note originally published on my Personal Blog here. Read original note so that you won’t miss any content.

Introduction

At last time I talked about snapshot testing on iOS. Now I would like to turn to the second, no less popular mobile platform, — Android. Here we are very limited in the choice of tools for the snapshot verifying using native frameworks.

The popular tool from Facebook Screenshot-Tests-For-Android will spend the biggest part of my note. Probably I should highlight at the beginning, that although this tool is the market leader and the trend for the forks, it grows rather slowly (v0.8.0 at the time of article publication) and it has several lacks, which I will discuss with a little comparative analysis with iOS.

We may have noticed the confusing difference between the type of the name of this testing on both platforms: Snapshot Testing on iOS and Screenshot Testing on Android. I propose to perceive it as a confrontation between a simulator and an emulator (:

Precondition

To pick up closer to Screenshot-Tests-For-Android project we need to make a few simple motions:

1. Install python dependencies:

$ pip install mock
$ pip install Pillow

2. Add permission for write in storage into AndroidManifest.xml:

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

3. Setup gradle plugin:

• in root build.gradle

buildscript {
   dependencies {
      classpath 'com.facebook.testing.screenshot:plugin:0.8.0'
   }
}

• in app build.gradle

apply plugin: 'com.facebook.testing.screenshot'

4. Create custom Test Runner:

Usage

In order to run screenshot testing along with simple Espresso tests, we will need to use gradle tasks:

Available checks:

• whole activity

• bloated view

• any element

Multiple checks:

If we would like to snap many screenshots in one test we should set up a name of every screenshot:

Multiple devices

In order to run screenshot tests on multiple devices, we will need to prescribe an extra property into app build.gradle:

screenshots {
    multipleDevices true
}

Our screenshots will be save in the unique directory with a composite name, e.g.:

API_23_GP_XXHDPI_1080x1920_armeabi-v7a_ru-RU

Tips of record

The only way to collect screenshots is this gradle task:

$ record<App Variant>ScreenshotTest

I am sure we don’t want to spend so much time to run all our espresso tests to collect all screenshots. It is not only about laziness, but also about her (:

So, let’s divide our tests into a screenshot-test and not a screenshot-test via one “if” statement in app build.gradle:

Reports

Here we came to the weakest part of Screenshot-Tests-For-Android. Yes, we can say that at the moment the report about failed tests is uninformative at all and does not show the visual differences with the actual snapshot and the reference one.

In pursuit of beautiful and understandable reports, we can come to one tool named Shot, which, inheriting Screenshot-Tests-For-Android, really “painted over” this problem. But finding awesome reports, we lose flexibility, because Shot can’t work with a lot of devices. Based on all of this, everyone can choose the extremes that will make his weekdays less painful.

Analize

In this block I would like to make a small comparison between hero of the article and his elder brother on “antagonistic” platform — iOS-Snapshot-Test-Case:

Conclusion

To sum it up, I would say that testing with screenshots on Android is pretty easy and useful. It is mighty speed up our automation testing and reduce sum of test cases.

If you had any questions or clarifications after reading the article about the screenshot testing, I’ll be happy to answer them. And you can find the completed project here:

https://github.com/alter-al/sample_of_android_screenshot_testing_2048

So, good luck, have fun, see you later (: