How to capture screenshots for failed UI tests
If you have any kind of tests then at some point they’re bound to fail. And usually it’s a good thing as they will detect bugs in your app 😃
The thing with Android instrumentation tests is that sometimes even though you have the stack-trace of what failed, it might be difficult to wrap your head around what’s the actual cause of failure. It’s even harder when your tests get executed on CI and you don’t see them failing right in front of your eyes so to speak. Also, sometimes they might fail on CI but run just fine on your local machine which might drive you crazy…
So wouldn’t it be great to see what was happening on the device when a given Espresso assertion failed and have that screenshot appended to your JUnit report?
Well, this article will show you how to do that exactly 😉
How to capture device screenshots in the first place?
Android Support Test library contains a Screenshot class which comes in very handy. Its capture() method uses UIAutomator under the hood (so Android 4.3 is a minimum) to get things done.
This is great as the screenshot will be taken even if the Activity is not currently displayed or if a system dialog is in the foreground.
According to JavaDoc capture() method:
Creates a
ScreenCapturethat contains aBitmapof the visible screen content for Build.VERSION_CODES.JELLY_BEAN_MR2 and above.
You can also change the name of the screenshot by using ScreenCapture#setName(String) to differentiate between different screenshots later on.
That’s great, but how to magically save this Bitmap on a device?
Don’t worry, we’ll get to how to download it from the device to your machine later.
Say hello to ScreenCaptureProcessor
In order to do something with your ScreenCapture you need to use a ScreenCaptureProcessor. There is a default implementation called BasicScreenCaptureProcessor which does the following:
A basic
ScreenCaptureProcessorfor processing aScreenCapture.This will perform basic processing on the given
ScreenCapturesuch as saving to the public Pictures directory, given by android.os.Environment.getExternalStoragePublicDirectory(DIRECTORY_PICTURES), with a simple name that includes a few characteristics about the device it was saved on followed by a UUID.
At this point in order to capture a screenshot (anywhere in your test code) you might use a function like this:
️IMPORTANT: Since we are saving on an external storage you need to make sure that you have WRITE_EXTERNAL_STORAGE permission added in the manifest (you can add it for a debug build only). When running on Marshmallow+ you also need to have those permissions granted before running a test. Thankfully, there is a useful property in AGP called installOptions which you can use in your build.gradle file:
-g is for granting permissions when installing the app (works on Marshmallow+ only) while -r is to allow reinstalling of the app. These correspond to adb shell pm install options. There are also other solutions available however this is the one we’re using in our project. Just be aware that this does not work with Android Studio yet.
This does the trick, however there are other things to consider…
Writing you own ScreenCaptureProcessor
Since we want to embed our screenshots into JUnit reports it would be great if they didn’t really have some random UUID in their name… Also, the default implementation puts those images in a screenshotsfolder under Picture on the device. Since other apps might be using it, it’s better to have our custom folder on the device. To achieve that we can create our own ScreenCaptureProcessor:
By extending BasicScreenCaptureProcessor we place new screenshots in a my_app_folder older on the device (obviously, replace that with whatever you want it to be called). So now, in order to take that screenshot we can use a method like this:
parentFolderPath is something we’ll use later to put screenshots of failed tests in a correct folder so keep on reading 😄.
How to detect that a test failed?
Write your own Rule
JUnit offers a TestRule just for that called TestWatcher which we can extend.
This takes a screenshot and puts it in a failures folder with a test class name subfolder. The name of the screenshot is also the name of the failed test method.
Now, if you had a test for an Activity called SampleActivity it could look something like this:
NOTE: There won’t be any screenshots if the application crashes 😠. If you don’t want crashes to stop all your remaining tests consider introducing Android Test Orchestrator in your project.
Make sure the screen is still there when taking a screenshot…
The sample above might work in most cases, however it does not take into account what ActivityTestRule does when a test finishes. According to the documentation:
If the Activity is running at the end of the test, the test rule will finish it.
So it is possible that when a test fails, the Activity gets finished (i.e. it disappears) and then we take a screenshot which now shows us device’s home screen… We’ve noticed this happening from time to time so we found a way to work around this.
Thankfully, we can use a RuleChain to solve this:
This makes sure that we take our screenshot BEFORE the teardown part of ActivityTestRule kicks in.
Gradle setup
By now we have our screenshots saved on the device, but how to retrieve them so that we could actually see them after our tests finish?
To solve this we need to create 3 new Gradle tasks — one for getting the screenshots, a second one for removing these screenshots from the device (we do not want to leave a mess behind) and a third one for creating the screenshot directory on the device in the first place (first task won’t work without it). Fetching screenshots should be automatically done after connectedDebugAndroidTest task for simplicity. To do this, in your app’s modulebuild.gradle add this:
To run tests and fetch the screenshots execute the following in the command line:
./gradlew connectedDebugAndroidTestThey should appear in a build/reports/androidTests/connected folder in your app’s module.
You can learn more about ADB commands here:
JUnit reports
We’re almost there so stay with me 😅
In order to have the screenshots embedded nicely in JUnit reports as shown on the image above we need to parse the JUnit HTML reports and add image tags below failed methods’ titles. Thankfully, these reports have a fixed format so we can easily write a Gradle task to handle this for us. The final version of our Gradle tasks looks like this:
embedScreenshotsTask above iterates over the screenshot failures. Since the file name of each screenshot corresponds to a failed method name and each folder name corresponds to a test class of that test method it’s easy to find an HTML report file for that test. Once the file is found, we simply add an <img/> tag in a correct place.
Show me a sample app
To put it all together I’ve created a sample app on Github which you can find here:
Hope you find this article useful & that it saves you lots of debugging time 😃 Keep on testing!
Read more about the technologies we use or take an inside look at our organisation & processes. Interested in working at StepStone? Check out our careers page.
