Accelerate your Android Espresso testing by grouping relevant tests
If you’re anything like me, you like it when there is one button to push that Just Does What I Want. When it comes to running tests in Android Studio, I like that it’s easy to configure it to run all my tests with the run button. However, when the number and complexity of tests grows over time, the actions behind that one button can take a lot longer than I’d like, especially if it’s testing application code that I’m not currently working with.
Why run all tests all of the time, when you really just want to run the ones that matter right now? Or at least just the ones that run quickly. Fortunately, the Android test tools give you a way to slice up your tests into related groups for faster testing. Here are your options.
Group by the “size” of the test
I put “size” in quotes here, because it’s not what you’d initially think. It’s not so much about the volume of code in the test, but what it does that could cause it to be slow. The Android test tools give you three size categories: small, medium, and large. To slot a test into each category, use one of the provided annotations (@SmallTest, @MediumTest, @LargeTest) on either an individual test method or the class that contains test methods (in which case, it applies to all of the contained test methods). Here’s an example:
@SmallTest
fun testSomethingSimple() {
// Test code...
}You can see here that the individual test method testSomethingSimple is annotated with @SmallTest. Judging from the name alone, it seems like a good candidate for a so-called “small test”, but what does that size category actually mean? Turns out, they have very specific descriptions, and I suggest clicking through to read the API documentation for each one. To summarize:
- Small tests don’t do any I/O at all, and should run less than 200ms.
- Medium tests can use local files and databases, but not networking, and should run less than 1000ms.
- Large tests can use everything, including local and remote data, to test full app integration.
If your tests launch your app and run through actual use cases, accessing data from remote APIs and databases, they all probably classify as large tests. In order to make a test classify as small or medium, you might have to look into strategies to fake, mock, or stub the code that does I/O. That can be a fair amount of work to set up if you aren’t already using some form of dependency injection. But I will say that, ideally, you probably want as many of your tests as possible to be “small”, even if it requires extra engineering effort. Stubbing, mocking, and faking remote data during testing can drastically speed up the feedback from your tests during development. However, implementing this effectively is a whole new topic!
So, let’s say you do have tests now categorized by size. In order to speed up your dev and test cycles, you might want to only run small tests, to get some quick feedback that your new code works, and that hasn’t immediately broken anything. What you’ll need to do is arrange to pass the a “size” option to the Android instrumented test runner using -e size small. You can add that to build.gradle under defaultConfig like this:
android {
defaultConfig {
testInstrumentationRunnerArguments size: 'small'
}
}But you probably don’t want to make this a permanent part of the build. Instead, you can pass this directive through via the Gradle command line like like this:
-Pandroid.testInstrumentationRunnerArguments.size=smallWith this, only tests annotated with @SmallTest will run. This is fine for grouping horizontally, by size, but what about vertically, by product feature?
Group by tests by feature (test suite)
Let’s say you’re working on just one feature in your app, and you know very well set of code being executed with that feature. Instead of running all the tests for your app, you might instead want to run just the tests that exercise that feature. One way to do that is with a test suite. A test suite allows you to define a group of test classes that should be executed together on demand, without having to run each one individually.
All you have to do is define an empty class, and give it two annotations, @RunWith and @SuiteClasses. Here’s a simple suite called TestSuite:
import org.junit.runner.RunWith
import org.junit.runners.Suite
import org.junit.runners.Suite.SuiteClasses@RunWith(Suite::class)
@SuiteClasses(Activity01Test::class, Activity02Test::class)
class TestSuite
@RunWith tells the test runner this is a suite, and @SuiteClasses lists all of the test classes that make up this suite. When the runner encounters this class, it will run the tests in all the named classes. You’ll have to arrange to use this one class just like you would any other single class, by passing -e class package.of.TestSuite01 to the instrumentation runner. You can add that to build.gradle under defaultConfig like this using its full class name:
android {
defaultConfig {
testInstrumentationRunnerArguments class: 'pkg.of.TestSuite'
}
}Or via the Gradle command line like like this:
-Pandroid.testInstrumentationRunnerArguments.class=pkg.of.TestSuiteThere is one thing to know about test suites. If you add a test suite class to your collection of tests, and choose to run all tests, your test suite will run in addition to all the other tests. Imagine a package layout like this:
What we have here is test classes defined in the package my.testapp, and a suite defined in my.testapp.suite that uses some of the classes from my.testapp. If you choose to run all tests for this project, or even all tests in package my.testapp, the suite will duplicate the effort of its named classes, and unnecessarily slow things down again. To avoid this, organize your suite and test classes into different packages that aren’t nested:
With this, we can choose to run all tests in package my.testapp.classes separately from the suite by passing -e package my.testapp.classes via build.gradle:
testInstrumentationRunnerArguments package: 'my.testapp.classes'Or the command line:
-Pandroid.testInstrumentationRunnerArguments.package=my.testapp.classesGroup by tests by other commonality (annotation)
Test suites are a good way to group tests by some common functionality, but it does require that you list them all out in a single class. This might not be very convenient, and long lists can be difficult to manage. It might be easier to instead mark each test class or method as part of a group to execute together. To do that, you can create a custom annotation, and use it to mark the classes and methods for execution. For example, here’s a simple annotation declaration:
package my.testappannotation class FeatureTest
Then, you can apply it to a test class or individual method:
@RunWith(AndroidJUnit4::class)
@FeatureTest
class Feature01Test {
// test methods here
}After you have classes and methods marked like this, you can then pass an argument to the test runner to indicate only these marked tests should run, using -e annotation my.testapp.FeatureTest. In build.gradle, you’d use this:
testInstrumentationRunnerArguments annotation:
'my.testapp.FeatureTest'And on the command line, like this:
-Pandroid.testInstrumentationRunnerArguments.annotation=my.testapp.FeatureTestNote that this method of using annotations is not mentioned in the formal documentation on the test runner command line, but you can see it in the API docs for AndroidJUnitRunner.
Summary
Grouping tests can effectively accelerate your development cycles by giving you a way to execute only the tests that matter to what you’re currently working on, or only the tests that you have time to run. It also could speed up the way you do testing in CI environments. For example, you might want to run only faster tests with each commit, and slower tests overnight. Or, on a feature branch, you might limit CI testing to only the feature under current development.
