The evolution journey of Android GUI testing

Cucumber — Espresso — Page Object Pattern

The Selfish Gene

History of test automation is full of countless test frameworks from all sorts of approaches. If you play for some time with many of them, eventually, you will start appreciating the patterns in those solutions that, shortly after, will determine their Darwinian survival. Ultimately, you will develop an instinct to recognize an emerging new solution as the clear species that will extinguish the rest from the so often rediscovered jungle of test automation and will create a new standard for future generations.

This is the story of those outliers that made a new point of no return in the evolution of testing.

The Behavior Chromosome

The first time I’ve experienced this enlightening moment was when I discovered Behavioral Driven Development or BDD. It was clear from the beginning that the proposal of BDD to express test scenarios in a common -subset of English- language shared among the three most important actors in any software development project (product owner, developer and tester), was the right choice to pursue in any project. The survival of BDD and its spread across the field -even to unit test frameworks- shows this was the right intuition.

This crystal clear executable example of BDD test talks by itself:

Feature: Login in the application

Scenario: User can login with valid user name and password
  Given I see the login page
  When I enter the user name "sebas" and password "passion"
  Then I see the welcome page

Why?

Clear expectations. When product owners, customers, testers and developers share the same expectations about how an application should look and behave, both the development speed and the quality of the product experience a drastic improvement.

She really wants to be free, let her go.

Large projects tend to accumulate piles of tests that nobody cares or knows what they are supposed to test. These tests are kept untouched like sacred cows to respect the code coverage, but the truth is that code coverage is good for code quality and low level bug squashing, while it does little to help build the right application or cover real use cases (especially when coverage comes mainly from unit tests). Like legacy code, legacy tests should also be re-factored and often deleted.

On the other hand, clear documentation is often confused with just lengthy specifications that most of the times bring more boredom and avoidance than clarity and consensus to the project’s team.

Therefore, we can easily arrive to the logical conclusion that clearly written expectations in the form of tests -even manual tests- essentially become the most up to date and relevant documentation of the application’s look and behavior. You may think that living documentation written in the form of automated tests sounds more like a magical fantasy than a truly evolutionary step, but the truth is out there and RSpec is one of the many great examples of executable documentation.

How?

Surprisingly easy. There are many BDD frameworks available for almost every language, in this article I will use Cucumber. This library simply transforms English written sentences contained in feature files into methods implemented in your preferred language -Java in our case-.

The best way to explain this is with an example that wraps your typical test function into a reusable and meaningful English sentence.

Here is a Cucumber feature file with a complete test scenario:

Feature: Login in the application
@ScenarioId("MyApp-135") @login-scenarios
Scenario: User can login with valid user name and password
  Given I see the login page
  When I login with user name "Sebas" and password "pasión"
  Then I see the welcome page
  And the title is "Welcome Sebas"

And here is the Java glue code for the last test step of the scenario above:

@And("^the title is \"(.+)\"$")
public void the_title_is(final String title) {
  assertTrue( title.equals(LoginViewModel.getTitle()) );
}

The first keyword in the sentence is interchangeable with: Given/When/Then/And

One important feature of Cucumber is that it allows you to mark your test scenarios with custom tags. It is very important that you tag your tests, mostly to trace features and User Stories of your product back to the tests that cover them. This has the extra benefit that executions can be filtered based on these tags (e.g. run only @login-scenarios).

Espresso: the perfect DNA combination for a deadly species

The second great evolution in testing came to me this year from the hands of Google, the father of Android. They decided to genetically combine the best of existing Android test frameworks like Robotium while solving the biggest problems GUI testing has in most common mobile platforms. Thus, Espresso was born into this world.

Why?

The biggest pain that all GUI tests suffer is slowness, mostly due to a very basic synchronization problem: when the test framework simulates a user action, it is not sure how long it will take until it is ready to accept more user input or when the information requested will be available to the user in the GUI, therefore, arbitrary waits have to be introduced in the tests to account for the delay between user request and system response.

When the new developer was asked to review the existing tests, this was his accurate description of current test automation.

Additionally, a worse side effect reveals itself long time after this problem was introduced: tests become flaky because the system unexpectedly behaves slower than normal and makes random tests timeout and fail without a clear root cause. This is the perfect reason for extinction of not just test scenarios but also for the whole test framework and eventually automated testing all together because developers and testers will silently run away from a gigantic and sinking test suite that everybody is tired of patching and nobody knows how to really fix.

Espresso elegantly solves this problem by synchronizing the test framework with the queue of GUI events that have to be processed by the system, thus removing once and for all the need to introduce arbitrary waits in tests, along with the main cause of test flakiness. The immediate consequence are blazing fast and very reliable automated tests.

Espresso tests run so fast that you won’t believe that all test scenarios have been really executed.

The other bonuses of the Espresso framework are:

• More readable, fluent and shorter code to perform actions and checks
• Easy to extend framework with a simple API

How?

To use Espresso include the library in your Gradle project using the official instructions.

Let’s see a simple Java example for the previous scenario:

@Given("^the title is \"(\\s+)\"$")
public void the_title_is(final String title) {
  onView(withText(title))
    .check(matches(isDisplayed()));
}

If you’re still stuck with an Eclipse project here is a library I made (plus example)

The beauty of a clean and readable code pattern

The last species I recently discovered wandering through the wild jungle of web development is the Page Object Pattern. At first sight, this species doesn’t look more interesting than an ordinary bird, but take a closer look and you’ll discover a magnificent beast of beautiful feathers that can impersonate all your GUI behavior with the grace of a peacock, no matter the size of your application.

As soon as your application reaches a few dozens of tests your test code is going to be full of helper methods that encapsulate how your tests communicate with your application and sooner than later the maintenance cost and code duplication of these helpers will cause you a lot of headaches. Here is where the Page Object Pattern will shine at its brightest.

To achieve such greatness this pattern creates a test interface that reflects the exact same behavior as the GUI is exposing to the user, and it is efficiently divided in the same pages and sub-pages (called Views, Activities or Fragments in Android) as the GUI does. Let´s see an example:

Imagine our Activity to Login into the application has the form of:

public class LoginActivity extends Activity {
  ...
  private EditText userName;
  private EditText password;
  private Button   loginButton;
  ...
}

Our tests will use this Page Object to interact with the tested application:

public class LoginPage {
  /**
   * Here goes your Espresso code to interact (actions)
   * with GUI ids and items concerning only this page/view
   */
  public WelcomePage doLogin(username, password) {
    onView(withId(R.id.user)).perform(typeText(username));
    onView(withId(R.id.pass)).perform(typeText(password));
    onView(withId(R.id.button)).perform(click());
    return new WelcomePage();
  }
}

This class will always encapsulate all the know-how to interact with the Activity but also the flow of the Activity after each action is performed, this is the reason why all actions always return either a new page object or the current object.

Why?

It is very reasonable to assume that the architecture of your tests should resemble very closely the architecture of your application but, contrary to common sense, this is usually not the case.

The most important reasons why the use of this pattern adds a lot of value to the project are:

• Clean and scalable architecture for large applications with large test suites.
• Single point of update when the application’s GUI changes.
• Encapsulate all implementation details in Page objects to focus testing on the what is being tested instead of how to test it.

How?

Read this short explanation of the pattern and start by separating all the test code in two parts:

• code that dictates what the tests should do: this code should be in feature files and Java glue code (also called step definitions)
• code that dictates how to interact with the GUI: this code should be in Page Objects

The example below puts everything together using the previous scenario:

@When("I login with user name \"(\\s+)\" and password \"(\\s+)\"$")
public void i_login_with_username_and_password(final String
  username, final String password) {
  (new LoginPage).doLogin(username, password);
}

The pyramid of life

Before closing I would like to point out that GUI testing is just the tip of the testing iceberg, literally. Although in the testing world we generally use the metaphor of a pyramid to represent the different types of testing, a pyramid just like the one below.

The biggest amount of test scenarios should be at the base with Unit tests first. Secondly, components and services which should have less test scenarios. And finally, GUI tests together with end to end tests should have the least amount of test scenarios, but, nevertheless, these tests represent the use cases with the highest business value.

All the solutions discussed in this article might not appear to give obvious advantages on small projects (less than 2k lines of code) and may even be a waste of resources on really tiny projects. In contrast, these solutions increase their value proposition exponentially with the size of the code and reach their fullest potential in large and challenging software projects.

A good developer is the one who is aware of different solutions and is capable of applying the solution that fits best each particular challenge.

For reference, this Android project includes a working example of all the testing solutions discussed in this article:

https://github.com/sebaslogen/CleanGUITestArchitecture