Use Espresso’s IdlingResource for max Android test speed
When you first start learning UI testing with Espresso, it begins fairly quickly and easily by coding using its fluent APIs that manipulate your app’s UI elements. For example, here’s a pretty straightforward test that launches an activity “MainActivity”, types in an email and password, presses a button to sign in, and checks to see if there is a welcome message on screen.
This sort of thing works well if the sign-in process is fast. However, if the sign-in happens slower than expected, perhaps due to network or disk access, the test can fail to find the welcome message. This is what you might call a “flaky test”: a test that doesn’t work reliably, depending on the current situation. The app being tested might be coded perfectly, but the test itself isn’t resilient to variance in performance. Your app and tests need to be coded to expect delays.
Nearly all Android applications today work with some data requested over the network. If not that, certainly data is stored in a local database, using libraries such as Jetpack Room. When an app works with data that requires some I/O, the user doesn’t get a guarantee for how long it’s going to take. We just can’t be certain about the speed of all the networks, CPUs, and filesystems involved. As such, your UI test code also shouldn’t make any assumptions, and instead, should anticipate delays in the execution of the app.
One immediate solution to this might be to make the test wait at the point of the delay to allow time for it to complete:
While this might work most of the time, it’s still not guaranteed to work all of the time. What if your network or login system are experiencing slowness? You could always bump up the sleep time, but that just makes your test that much longer to complete in normal conditions. Forcing a wait time like this can have a huge impact on the total time it takes to run your test suite — don’t do this!
Instead, it might be tempting to check periodically if the welcome message becomes visible. This could be done by putting a loop around the check, and ignoring the exceptions thrown when the view isn’t visible yet:
Yes, that code is just as ugly as it looks. Don’t do this either. While it might find the view as soon as it becomes visible, it’s also chewing up CPU time on the device where it’s running, which could make the app run slower while it’s under test. This code also doesn’t have a sense of timeout, so if there’s an error, it won’t know when to terminate. You’d have to add more lines to make sure this looping only goes on for so long. Yuck. Fortunately, there’s a way out.
Use an Espresso IdlingResource instead
A better way to handle this is using an Espresso IdlingResource. An IdlingResource is an object that lets your app code tell your test code when the app is busy with some background work, so that the test code waits until that work completes. It’s pretty easy to understand, but has some challenges to implement. In general, it works like this (I’ll use “IR” as shorthand for IdlingResource in this post):
- The app exposes an IR to the test code via the activity under test.
- The test registers the IR and pays attention to its state changes.
- The app indicates to the IR when it’s occupied with work (e.g. making a network request, or loading data)
- The test pauses when the IR is “busy” and resumes when the IR is “complete”.
- The test unregisters the IR just after the test is complete
Basically, the IdlingResource is a way for the app to tell the test that it’s busy with some asynchronous work, then again when that work is complete. This lets the test wait for the app to finish any background work, avoiding the need to code a sleep or a loop that can waste time and make your test difficult to understand.
The tricky part is getting that IR from your app code into the test. The Espresso documentation on IR gives you some options. The recommended approach is to have your Activity expose the object directly to your tests.
If your test is working with an Activity object directly, and it exposes an IR directly from a property, test code starts to look something like this:
You can see above that the @Test method is unchanged. However, @Before the test executes, the code will get a hold of the Activity being tested, cast it to the expected type, and reach into it to get the IR. The IR is registered with Espresso. @After the test is over, it’s unregistered. There is nothing else to be done here — the test will automatically pause when the IR indicates that the Activity is busy, then immediately resume when it’s idle. All that remains is using the IR correctly in the app’s code to indicate status. Here’s how.
You can count on CountingIdlingResource
Espresso provides some implementations of IdlingResource. The most common is CountingIdlingResource, and others are derived from it. The way it works is pretty straightforward. It’s governed by these rules:
- It starts with an initial “count” of 0, the number of ongoing background tasks.
- App code can increment or decrement the count to indicate when a task starts or stops.
- A count of 0 means the app is “idle” — no work is ongoing.
- Anything greater than 0 means the app is “busy”.
What you can do is create an instance of CountingIdlingResource, expose it to the test through your Activity object, and work with it in your code. It sounds simple, but requires some architectural planning to make this work well. If you’re using MVVM, an easy place to work with an IR is in your ViewModel, which manages the calls to your repository objects that are doing the actual work.
This can get complicated, and there are a lot of ways to do this. As a super brief example using Kotlin coroutines and LiveData, say you have a ViewModel that signs in the user using a suspending function on a repository, and returns a LiveData to the UI layer:
Some details are omitted for brevity, but you can see here that:
- A CountingIdlingResource is created, named, and made available to the UI layer by a public property
- The IR is used to indicate when the sign-in process is initially busy by incrementing it
- The IR is then marked complete by decrementing it, even if an error occurs
The UI layer (Activity) can then reach into the ViewModel to get the IR, then pass it along to the test using another public property.
(For very clean architecture, you might want to inject the responsibility of working with IdlingResources instead, from the test code itself.)
By default, Espresso will wait 1 minute for an IdlingResource to transition to an idle state before timing out the test, which might not be what you want. If that’s too long, you could waste time on a test that’s expected to fail faster. Or, if it’s too short, your test could become flaky. To change the default, you should set the timeout @Before your test starts using IdlingPolicies:
IdlingPolicies.setMasterPolicyTimeout(10, TimeUnit.SECONDS)
IdlingPolicies.setIdlingResourceTimeout(10, TimeUnit.SECONDS)If you like IdlingResource, also try BusyBee
One last thing. If you make heavy use of IdlingResource, you’ll find that becomes more difficult to use as your app has more complicated tasks to perform. Consider replacing them with BusyBee, developed by Michael Bailey at American Express. This utility makes it easier for you to group and debug your tasks. It can also be used outside of Android modules. Maybe best of all, it uses a singleton to share IdlingResources with Espresso, eliminating the need for your test to reach into an activity to find their instances.
Whether you use IdlingResource or BusyBee, your tests should always expect delays, but minimize the amount of time spent waiting. It takes some engineering work to instrument the app, but it’s worth it.
