Android Instrumentation annotated tests
How to group instrumentation tests in different buckets
At Gumtree we truly believe in Automation Tests. Automation Tests help the team to move faster, giving us more confidence whenever we create new features and/or refactor our code.
Automation Tests suits can execute thousands of different test cases every time they run, providing coverage that is infeasible with manual tests.
We have a ton of Unit Tests to cover the business logic of the android app. Unfortunately, Unit Tests do not give us the whole picture. We want to be able to automate all those Use Cases that a QA engineer would test manually to make sure the app is working as expected.
Fortunately for us, Instrumentation Tests allow us to test all those Use Cases. That’s why we’ve been lately investing some time working with Espresso in order to create what we internally call Use Case Tests.
Our Use Case Tests are simply Instrumentation Tests covering some specific Use Cases that we’ve defined to make sure a feature passes all the Acceptance Criteria.
Use Case Tests galore
As the code has been growing with new features, Use Case Tests have been growing with them along the way. That’s indeed a very good sign! Unfortunately, unlike Unit Tests, Use Case Test take a significant amount of time to run.
The idea to run all the Use Case Test every time there is a change in the code sounds amazing, but in practice, it was not helping us to move fast. In our project, running all the Use Case Tests could take a bit over 10 minutes, which was not very practical when there was just a small change.
That’s why we’ve decided to use a different strategy. The idea is to group our Use Case Tests in different buckets so we can decide when is the more convenient time to run a specific group of tests.
What are Test buckets for?
That is totally up to you!
Imagine you have some core tests that you always want to run before commiting, or some tests that you just want to run when a development flag is turned on, or maybe some other group of tests that you want to run when working on a specific feature.
We could think on a lot of different scenarios in which it could make sense to have different tests grouped into buckets. The main idea is to give us the flexibility to decide when is more convenient to run a certain group of test.
Ok, I’m in. How complicated is that?
Fortunately for us, it is very easy to set this up in Android. As an example, let’s imagine we want to put a group of Instrumentation Tests in a bucket called CoreTests.
1. Create a CoreTest annotation
The first step is to create a custom CoreTest annotation.
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface CoreTest{
} 2. Mark Instrumentation test as CoreTest
After that, we add the @CoreTest annotation in the Instrumentation Test that we want mark as CoreTest. This annotation can be used at a method or class level.
@RunWith(AndroidJUnit4.class)
@CoreTest
public class PostAdUseCaseTest{
...
}3 . Run CoreTest bucket
Once the custom annotations are in place, we just need to know how to run the annotated tests.
- In Android Studio
In the Run/Debug Configurations Window, add the following extra option:
-e annotation com.your.annotation.package.CoreTest
- Using Gradle
./gradlew app:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.annotation=com.your.annotation.package.CoreTestThat was easy!
Now you know how to easily group your Instrumentation Tests into different buckets.
One of the main advantages of this technique is that it allows us to put more than one annotation in each class/method. That means that the same test could be part of two or more different buckets at the same time. How cool is that?
I hope you found this post helpful and you enjoyed the reading.
Please, write us any comment and/or suggestion and do not hesitate on letting us know which buckets you’ve created to group your Instrumentation Tests.
