The Resurrection of Testing for Android

An oft-overlooked aspect of development gets a fresh look.

Square Engineering
Apr 3, 2013 · 5 min read

Written by Jake Wharton.

Heads up, we’ve moved! If you’d like to continue keeping up with the latest technical content from Square please visit us at our new home https://developer.squareup.com/blog

When testing Android applications, documentation and tools can be sparse. At Square, verifying the behavior of our applications is an essential part of our day-to-day development — and to make testing easier, we have created and refined a set of tools for three different aspects of testing; unit tests, instrumentation tests, and declaring assertions.

Robolectric

The most fundamental problem which inhibits testing is that the API jar for each version of Android is empty. Rather than having actual code, the provided jars are filled with code that looks like this:

As you can imagine, having every constructor and every method in the entire platform immediately throw an exception prevents any real tests from being written.

Robolectric is a framework that has been designed to solve this problem by replacing these “stub” jars with the actual code. This means that any behavior you want to test will flow through the same code paths as if it were running on a real device.

Square continues to improve Robolectric and the forthcoming version 2 has significant ease of use and major performance improvements. Let’s write a quick test to demonstrate how easy it is to add to your project.

Given an ImageView that is designed to keep equal width and height, we can verify that it correctly exhibits this behavior.

As mentioned before, if we were to execute this as-is we would immediately see a RuntimeException thrown. Let’s add Robolectric to our project and modify our code to use it:

First we add an annotation on top of our test class that we previously created. This annotation tells JUnit that it should use Robolectric in order to execute the test. Behind the scenes Robolectric will load the appropriate Android runtime as well as all of the resources from your application.

Were you expecting a second step? Turns out that for most cases, the annotation is all that we need! We can now do all of the things that you would expect when writing tests: layout inflation, view and activity creation, and UI interaction.

For more information about Robolectric please visit its website and blog. Version 2.0 is currently in alpha but has been fundamental in enabling effective testing at Square.

Spoon

While unit tests are useful for testing everything from UI behavior to complex, low-level internals of your application, sometimes it is useful to verify behavior at a higher level. Instrumentation tests were designed to address this in more of a black-box model.

Unfortunately instrumentation test execution and verification is still a very manual process. Spoon was designed to address this by automating the test execution across multiple devices as well as aggregation of the results.

Plug in as many devices as adb will support and fire up Spoon using your existing application and instrumentation APKs. Here we see the sample app from Spoon — a variation of Roman Nurik’s wizard flow — running on four devices at once. Each device has a different screen size and version of Android in order to get more useful results.

Where Spoon really shines is in its aggregation of screenshots while your tests are running.

The result of adding screenshot support to your tests is visual insight into your application’s behavior across multiple devices.

In addition to static screenshots you can also view each test in animated GIF form.

You can learn more about Spoon from its website. Sample output for the provided demo tests is also available to browse.

FEST Android

Despite the undeniable usefulness of tests, they are not always fun to write. An even less enjoyable experience is debugging why a test is failing. Built upon FEST, FEST Android is a declarative, fluent API for verifying the state of Android objects.

In the examples above we wrote three assertions using the methods provided by JUnit:

While these are relatively easy to read, the exceptions that they throw when failing are not very helpful.

While this message is not too bad, it does not describe exactly what is being verified. The exception message for the login assertion is even worse.

These messages report the problem in its raw form but provide no context for what we were verifying or what needs done. Let’s rewrite the assertions using FEST Android.

Immediately we are already given a much more declarative syntax that reads almost like English. Not only are these easier to write, but when looking at a test from another developer you can understand what is being tested much more quickly.

When failing, the exceptions provide much more context into the actual problem.

With these more informative messages it has been our experience that most times we can dive directly into the code to fix the problem rather than first having to figure out what broke.

More examples of usage — as well as download links — are available on the library’s website.

Triple Threat

The three aforementioned libraries enable us to efficiently write and execute tests for every aspect of our Android code. The result of which is increased confidence in our product and ability to iterate and ship more rapidly.

So while Android may not be the most forgiving framework when it comes to enabling you to write tests, our hope is that the libraries and tools we provide help break that trend.

P.S. There are also a few other great libraries for testing that augment everything described above. Some of our favorites are Mockito, FEST, Robotium, and MockWebServer.


Square Corner Blog

Buying and selling sound like simple things - and they…

Square Engineering

Written by

The official account for @Square Engineering.

Square Corner Blog

Buying and selling sound like simple things - and they should be. Somewhere along the way, they got complicated. At Square, we're working hard to make commerce easy for everyone.

Square Engineering

Written by

The official account for @Square Engineering.

Square Corner Blog

Buying and selling sound like simple things - and they should be. Somewhere along the way, they got complicated. At Square, we're working hard to make commerce easy for everyone.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store