Android Testing: How to Perform Instrumented Tests

In the previous posts, I wrote about the main considerations to be kept in mind when structuring the app to be easily testable and identified the main concepts and tools.

We also got to work and implemented a series of unit and integration tests using the most common Mockito tools and functions.

To finish this series of posts, we will now see the so called instrumented tests, which is the basis for the UI tests.

In addition, as we already mentioned in previous posts, it is also a great tool for End-to-End tests and any other test that requires working with the app as a whole.

Dependencies

These are the dependencies we will add the build.gradle file of the app module for the examples provided below.

androidTestImplementation 'androidx.test:runner:1.1.1' androidTestImplementation 'androidx.test:rules:1.1.1' androidTestImplementation 'androidx.test.ext:junit:1.1.0' androidTestImplementation 'androidx.test.espresso:espresso-core:3.1.1' androidTestImplementation 'com.nhaarman.mockitokotlin2:mockito-kotlin:2.0.0' androidTestImplementation 'org.mockito:mockito-android:2.23.4'

In order to continue using Mockito-Kotlin in this kind of tests on the Android VM, it is necessary to add the Mockito-Android dependency.

The other dependencies reference the runner, which in this case will be AndroidJUnit4, and Espresso, the framework used for instrumented tests.

To ensure compatibility of the runner we will use, make sure you have this configuration included in the build.gradle file, referencing the “ androidx” package (not the support):

android { //... defaultConfig { //... testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" } //... }

Note: Since the release of Android Jetpack, the various support packages have been retrofitted and included under the androidx package.

The example screen

We will work with a very simple screen, representing the famous MainActivity.

The layout consists of the following elements:

• EditText (id: myEditText)
• TextView (id: myTextView)
• Button (id: myButton)

We will use MVP to structure it, consisting of:

• MainView: interface implemented by MainActivity.
• MainPresenter: interface implemented by the MainPresenterImp
  class.

In addition, we will create a use case:

We define the following functional requirements:

• The initial state must be as follows:
• The EditText must be empty.
• The TextView must show the text “HelloWorld”.
• The button must be active.
• The following should occur when the button is clicked:
• The use case must be called to get some text from the value entered
  in the EditText.
• The value returned by the use case will be returned to the TextView,
  replacing its previous value.
• The content of EditText must be cleared.
• The button must be deactivated, preventing any future click.

For simplicity of this example, we will initialise the presenter during onCreate, although it would be best to inject it. In any case, as we give it public visibility, we can replace it for our tests.

Simplifying the classes are left as follows:

interface MainView { fun getEditTextValue(): String? fun cleanEditText() fun getTextViewValue(): String? fun setTextViewValue(value: String) fun disableButton() fun isButtonEnabled(): Boolean }
class MainActivity : AppCompatActivity(), MainView { lateinit var mainPresenter: MainPresenter override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) setContentView(R.layout.activity_main) mainPresenter = MainPresenterImpl(this, GetTextUseCaseImpl()) myButton.setOnClickListener { mainPresenter.onButtonClick() } } override fun getEditTextValue(): String? = myEditText.text?.toString() override fun cleanEditText(){ myEditText.text = null } override fun getTextViewValue(): String? = myTextView.text?.toString() override fun setTextViewValue(value: String) { myTextView.text = value } override fun disableButton() { myButton.isEnabled = false } override fun isButtonEnabled(): Boolean = myButton.isEnabled }
interface MainPresenter { fun onButtonClick() }
class MainPresenterImpl(private val mainView: MainView, private val getTextUseCase: GetTextUseCase) : MainPresenter { override fun onButtonClick() { val output = getTextUseCase.getText(mainView.getEditTextValue()) mainView.setTextViewValue(output) mainView.cleanEditText() mainView.disableButton() } }
interface GetTextUseCase { fun getText(input: String? = "no text"): String }
class GetTextUseCaseImpl : GetTextUseCase { override fun getText(input: String?): String = "This is the UC result for '$input'" }

Creating the Test class

The skeleton of a Test class for instrumented tests is very similar to the one we saw for unit tests in our second post.

Again, we have the annotations @Before, @Test and @After, and we continue to be able to create and inject mocks as required.

However, in this case we need the specific Runner for this type of tests using a class annotation @RunWith and a field annotation @Rule that allows us to define the Activity we are going to work with:

@RunWith(AndroidJUnit4::class) class MainActivityUiTest { @Rule @JvmField var mActivityTestRule : ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java, true, false) @Before fun setUp(){ val intent = Intent() //Customize intent if needed (maybesome extras?) mActivityTestRule.launchActivity(intent) } @Test fun someTest(){ //... } }

There are alternative ways to structure an instrumented test and, in fact, the one created by default on creating a new project is somewhat different.
However, the one I present is the way described in official UI test documentation.

The ActivityTestRule we have defined has 3 parameters:

• The first indicates the Activity class to be executed.
• The second indicates if the activity should be configured in “touch
  mode” on start.
• The third indicates if the activity must be re-launched automatically before each test.

You may want to set this last parameter as true. For the example, I thought it best to show you how to manually start the activity in the setUp(), so that we can use a custom intent.

This allows us to add “extras” and simulate multiple scenarios depending on the activity input data, when its behaviour depends on them.

And what if we want to

Test a Fragment?

In this case, you should start the Activity that contains it in the same way. Assuming the Fragment is loaded at the start of the Activity, you are ready.

If on the contrary, you need to load a different Fragment; getting to it will depend on your navigation architecture. You can manually run this navigation or perhaps condition the Fragment loaded initially depending on the Bundle received in the intent, which we have already seen is something we can edit for the tests.

In any case, bear in mind you can work with the Activity as much as you need to prepare the scenario before executing the test, although this implies a prior navigation step.

Controlling data

When working with UI tests, it is advisable to isolate the presentation layer from the rest of layers, which are not really the ones being put to
test.

We can see these as integration tests between the view and its presenter and any other controller class, such as for example the ViewModel if you use the architecture components of Android JetPack.

In any case and within this example, it would be enough to mock the use case bridging between the Presenter and the domain layer, and mock how the data is returned.

As already mentioned, the Presenter that the Activity works with, despite not being injected, is accessible, hence we have the possibility of substituting it in the setUp, just after launching the Activity.

@RunWith(AndroidJUnit4::class) class MainActivityUiTest { @Rule @JvmField var mActivityTestRule : ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java, true, false) //Collaborators lateinit var getTextUseCase: GetTextUseCase @Before fun setUp(){ val intent = Intent() //Customize intent if needed (maybe some extras?) mActivityTestRule.launchActivity(intent) val activity = mActivityTestRule.activity System.setProperty("org.mockito.android.target", activity.getDir("target", Context.MODE_PRIVATE).path) //needed workaround for Mockito getTextUseCase = mock() whenever(getTextUseCase.getText(any())).thenReturn("This is the UC mock result") val mainPresenter: MainPresenter = MainPresenterImpl(activity, getTextUseCase) activity.mainPresenter = mainPresenter } @Test fun someTest(){ //... } }

We can get the Activity instance that has been launched from the ActivityTestRule and, later, substitute the Presenter for it to use a UseCase mock.

Now, the test scope is exclusively limited to the behaviour of the view and its presenter, and we could therefore change the data type returned or raise exceptions and verify that the screen displays the error to the user as expected.

Note: as at the date of writing this post, operating with the Mockito version for Android VM (directly or through Mockito-Kotlin) causes an error indicating we must set the “org.mockito.android.target” system property. The issue has been registered in the Mockito GitHub and seems it will be corrected in upcoming versions without having to add it manually, but in any case, we solve the problem with this line.

How to interact with the view

The major difference between how we work a function for a unit test and how we do it for UI test comes now.

onView(…).perform/check(…)

In UI tests we first need to identify the view on which we will operate (first block) and, later, interact with it (second block), either executing an action on it or simply checking its state.

Although we may find more complex scenarios, we will almost always work with these two blocks.

For example, let us assume we want to click on the button in order to later verify other view states. We could achieve this as follows:

@Test fun someTest(){ onView(withId(R.id.myButton)) //first block .perform((click())) //second block }

The first block returns an object ViewInteraction and expects Matcher as its parameter. We use withid (search by id) in the example but there many others such as withText that allow you to search a view by its text, either for the String resource ID or the String itself.

Matchers can be combined in pairs and cascaded, so as to search the view the matches all of them:

onView( Matchers.allOf( ViewMatchers.withText(R.string.textResId), Matchers.allOf( ViewMatchers.isDescendantOfA(ViewMatchers.withId(R.id.ascendantId)), ViewMatchers.isDisplayed() ) ) )

When we want to validate a condition on this view, just change the second block now using the “check(…)” function, which expects a ViewAssertion as input. It is very easy to create one through the Matchers as follows.

onView(ViewMatchers.withId(R.id.viewId)) .check(ViewAssertions.matches(ViewMatchers.isDisplayed()))

First test

Back to work!

We will create a test to verify that when the Button is clicked, the text returned by the UseCase is displayed in the TextView.

In addition, we know that the text must be mocked in the setUp: “This is the UC mock result”.

@Test fun whenButtonIsClickedTheUseCaseTextIsShown(){ onView(withId(R.id.myButton)).perform((click())) onView(withId(R.id.myTextView)).check(ViewAssertions.matches(withText("This is the UC mock result"))) }

As you can see, the “withText” Matcher, just like any other, can be used in any of the two blocks, either to identify a view or to carry out a check on it.

On execution, it will ask us to select the device (real or emulated) and we will see live how the operations configured in each test are carried out. In addition, you can see how the Activity is re-launched for each test without preserving any state from the previous execution.

As it could not be otherwise, the test passed correctly. We will now create a few more:

@Test fun whenButtonIsClickedTheEditTextIsCleaned(){ onView(withId(R.id.myButton)).perform((click())) onView(withId(R.id.myEditText)).check(ViewAssertions.matches(withText(""))) } @Test fun whenButtonIsClickedItIsDisabled(){ onView(withId(R.id.myButton)).perform((click())) onView(withId(R.id.myButton)).check(ViewAssertions.matches(not(isEnabled()))) }

The first checks that the EditText is empty and the second that the button is not enabled. The notation is quite descriptive and I think it does not need much explanation.

We will now mix the power of Espresso and Mockito in order to also verify the graphical behaviour, resulting from correctly calling the UseCase (and not, for example, the presenter painting a hard coded value).

@Test fun whenButtonIsClickedUseCaseIsCalledWithTextFromEditText(){ onView(withId(R.id.myEditText)).perform(replaceText("Test text")) onView(withId(R.id.myButton)).perform((click())) val captor: KArgumentCaptor<String> = argumentCaptor() verify(getTextUseCase).getText(captor.capture()) assertEquals(captor.firstValue, "Test text") }

We can also verify that the initial states set in the requirements are met on starting the Activity, or how we treat a possible exception thrown from the UseCase, but as it does not require any element that we have not already seen in this or any of the previous posts, allow me to skip over it.

On a graphics level, our example is so simple that there is not much to test, although of course a more complex view may require some additional tools.

Related views

Sometimes we need to verify that a view we are going to operate on
is related to another view, such as for example a Toolbar or a Dialog.

This is especially useful if we do not know the ID or simply prefer
to maintain a “pure” black box mentality and perform all searches by
value and not by identifier. Two examples.

First, imagine you want to
verify the “Detail” text of a toolbar being displayed, in order to
verify for example that are seeing a new screen.

For this, we will add a Matcher in the following example indicating that the view we want to work with must be a descendant from another view with the identifier R.id.toolbar:

@Test fun thisIsATest() { //perform some operation over some view... onView( Matchers.allOf( ViewMatchers.withText("Detail"), ViewMatchers.isDescendantOfA(ViewMatchers.withId(R.id.toolbar)) ) ).check(ViewAssertions.matches(isDisplayed()))

Two, suppose you want to check that a text “This is a Dialog” is being displayed in a view whose root is a dialog. We could have something like:

@Test fun thisIsATest() { onView(ViewMatchers.withText("This is a Dialog")) .inRoot(RootMatchers.isDialog()) .check(ViewAssertions.matches(isDisplayed())) }

In this latter case, we change the scope of the verification to the root of the view with the text “This is a Dialog” ( inRoot function), also indicating it must be a dialog.

If the check confirms it is effectively being shown, it means this dialog with this text is present on screen in one of its elements.

There is a broad variety of cases, but as it is an introductory post I encourage you to read the documentation and try other options!

Navigation tests

This type of tests are perfectly valid for verifying that in the event of
various conditions, the app navigation performs correctly, both when going forward and when going back.

After all, we have a complete version of the app install in the device and we can interact with it as much as we like.

Following the black box principle, a fairly well accepted approach to verifying you navigated to the right view is not to check the current activity or fragment but to verify some of its visual elements (for example the title of the toolbar), and we can perform this kind of checks with the tools that we already know.

End to End (E2E) tests

In the example we have been using, we made use of the setUp function in order to substitute the Presenter and mock the UseCase. This created a controlled scenario in which only the View and the Presenter were being tested, without involving the rest of classes and layers that take part in the actual app.

Well, if we so wish, we cannot create any kind of mock on any element and launch the app with an intent identical to the one that would be used in the real scenario, without making any other modifications.

In that case, we would already be working with the actual app, even with the network calls consumed by the corresponding web services, putting the whole system to the test.

In short, we can imagine it as if a member of the team manually installed the app, started it and you start to interact with it in order to check that everything is OK, only that in this case the process is done automatically. As you can imagine, the time we save throughout the project is immense.

We can even take advantage of these tests to take screenshots and
record videos that allow us to later, at a glance, check there are no undesired
mismatches, especially when these tests are executed on multiple devices.

If you are not sure, it will be interesting to know that Android Studio has a very useful for recording Espresso tests (Run > Record Espresso Test) detecting the interactions that we perform on a test device and automatically generating a test function with all of them.

Any checks we want to perform must be manually added to the correct point in the function, but we will have a good skeleton for “simulated user” testing, in which we may want to perform a very complex navigation through the app.

Simplifying Kotlin nomenclature

Although it is true that the nomenclature of the various Espresso functions is in itself very easy to understand, it is also true that in more complex view we may have functions with too much “ boilerplating”, when in reality the key elements are very few.

This is accentuated for navigation tests or the other examples we mentioned in the previous section, where you may want to simulate dozens of interactions in a single function.

Thanks to Kotlin and, specifically, a mixture of its infix and extension functions, in the last project I worked in, we defined functions that considerably simplified the reading of this type of tests.

I think they can be very useful, so let me show you some examples so that you will be able by yourselves to create all the ones you need.

Utility functions:

infix fun Int.perform(action: ViewAction) { onView(ViewMatchers.withId(this)).perform(action) } infix fun Int.checkThat(matcher: Matcher<in View>) { onView(ViewMatchers.withId(this)).check(ViewAssertions.matches(matcher)) } infix fun Int.checkThatTextIs(text: String) { onView(ViewMatchers.withId(this)).check(ViewAssertions.matches(withText(text))) } infix fun Int.replaceTextWith(text: String?) { onView(ViewMatchers.withId(this)).perform(ViewActions.replaceText(text)) }

Refactoring the tests (we also import .R.id.*), we have something as follows:

@Test fun whenButtonIsClickedTheUseCaseTextIsShown() { myButton perform click() myTextView checkThatTextIs "This is the UC mock result" } @Test fun whenButtonIsClickedTheEditTextIsCleaned() { myButton perform click() myEditText checkThatTextIs "" } @Test fun whenButtonIsClickedItIsDisabled() { myButton perform click() myButton checkThat not(isEnabled()) } @Test fun whenButtonIsClickedUseCaseIsCalledWithTextFromEditText() { myEditText replaceTextWith "Test text" myButton perform click() val captor: KArgumentCaptor<String> = argumentCaptor() verify(getTextUseCase).getText(captor.capture()) assertEquals(captor.firstValue, "Test text") }

You can create as many functions of this type as you want, depending on the type of most common operations in your tests. In the end, you will have
something as simplified and easy to read as we have seen.

State and thread management

We will try to answer the following question, how do instrumented tests actually work?

The truth is that when we are executing one of these tests, what really happens is that the test app is being installed with an additional app,
which is in charge of executing your app and operating on it to run the tests
(we shall call it the controller app).

This is a very important consideration, given it indicates we have an app (test app) with its own Main Tread, and the controller app, with a different Main Thread.

There are two things to consider when we perform any operation on a view through Espresso:

• The controller app waits for the Main Thread of the test app to be idle and does not continue until this is the case.
• After detecting this state, the controller app queues the task in the Main Thread of the app being tested and, as expected, continues its executions.

With regards to the first point, there are certain elements that can lead the test app to never reach that state, such as the animations (for example a ProgressBar rotating continuously waiting for something to happen).

This may cause the test to stop at a certain point and end failing. From what I have seen, this type of blocks vary even depending on the Android version we are testing with.

Hence, the Espresso documentation advises we deactivate any device animations we use in order to run the tests.

There are some automated proposals to deactivate animations before executing a test (Gradle configurations or certain Rules) that are not quite
producing the expected results in all devices, and the truth is that the only
“infallible” solution is to follow the advice provided in the official documentation and manually deactivate all animations.

With regards to the second point, as these are two processes running
asynchronously, it is normal to find Espresso performing some checks on a view without it having given enough to time complete the execution of a task in the test app.

Following this example, clicking on the button calls the UseCase, which runs a certain process in background. The response is received when this process completes and the TextView displays the result. Once this flow is finished, the button is disabled.

For the test, we perform a “perform click” on the button and immediately after check it is not enabled. It is very likely the test will fail because the button has not yet been disabled when checking this condition.

In order to try to control this asynchronous execution, Espresso provides the IdleResources, which you can configure a wait period to continue with subsequent actions and checks.

There are also other patterns based on retries with timeout or even adding custom processes in the architecture.

In any case, as this is an introductory post, I think it is enough to understand this and be aware of these possible failures or apparent incoherencies. If you find yourself in this scenario, you know where to start looking.

Ready, set, go!

By now we can say you are fully initiated in this Android app oriented testing thing, with knowledge ranging from the initial design of the app architecture to the development of unit, integration, UI and E2E tests.

If you have really been able to put all these concepts and tools to the test, which as we have seen are not that many, I assure you that you are more than ready to face the vast majority of tests you will need in any of your developments.

It is now up to you to put this knowledge into practice, expand on them and become a real pro! Good luck!

Originally published at https://en.paradigmadigital.com.