How we run automated UI tests for our native iOS apps at Stuart

Stuart’s Apple pie recipe

Every chef has their own recipes. An apple pie can be made in many styles, all tasty in their own ways… but there is one special recipe that we love the most. Let me show you Stuart’s testing framework recipe that the QA team built in order to perform UI tests on our iOS apps.

I’ll talk you through the apples, the baking tin, and the flour we chose.

The Apple

The first ingredient I want to talk about is the testing framework, our “apple”. There are several options such as Appium, EarlGrey, Calabash but we went for XCTest.

But why?

1. This is a framework supported by Apple (and we know that they are putting effort to make it better year after year);
2. It would be fully integrated in the app code. This opens a door to our iOS developers to add/fix tests and is a way to encourage them (and of course, the QA team!) to add tests for every feature or bugfix in the same pull request.
3. Last but not least, we can share our mocks between UI tests and unit tests which makes writing new tests really smooth.

The Baking Tin

After selecting the proper apples, we want to make the pie a great shape. I personally hate it when I cut a slice and it breaks so the Stuart Apple cake will have to resist changes made and keep looking amazing. Before we start doing anything we have to breathe and think which architecture we want to follow in our project. We want to keep it simple while making the code reusable and easy to maintain.

In order to fit those needs, the page object pattern will be used to model the screens of the app. We’ll also use a robot pattern in order to have all the interactions with the application encapsulated in a single place.

The tests will use the screens to interact with the app and the screens will use the robots to perform actions (i.e. find elements, get text, tap, swipe… ).

Additionally, we created a different application target to decouple the application from the tests.

This allows us to create mocks in the application target (and in some other files) to define default values for those mocks. For example, we have a mockClient (defined in a file that has the app target) that will have the firstName John and lastName Doe. These variables are defined in a file that is shared for both targets and only contains static data, so we can refer to the data even if we want to perform an assert in the test target (or even build a mock with it).

The Flour

The pie is almost ready to go into the oven but we are missing the last (and maybe the most important) ingredient: the flour. A Regular flour can do the job but a good one makes the difference.

What am I talking about when I say “flour”?

We don’t want to hit the network with our tests! The objective of these tests is to check that the UI of our application behaves in a certain way after a certain interaction. What if a test fails because our API returns something valid but we are expecting something slightly different? What if the network request takes too long? The test might fail. We don’t want to receive an annoying notification every time that a test fails because of a network request or (even worse) get used to having failing tests.

We did some research about what can allow us to achieve that goal. The options we found interesting were OHHTTPStubs, Embassy/Ambassador, MockServer. Each one has pros and cons but… our final decision was none of these!

The main reasons are that we don’t want to add external dependencies to our project and/or maintain a lot of API responses. So we used a different approach.

Let’s go step by step. We want to ensure that our APIClient does not hit the network. Let’s add a few lines of code in our APIClient that will do the work and initialize our URLSession:

But… how do we know if the test isUITest and what is MockUrlSession? Be patient my friend, the details are coming!

In order to let the app know if we are running tests, let’s use an awesome feature that Apple provides us: launchArgument and launchEnvironment. Before each test execution we’ll launch an argument indicating that we are running tests. This method in the Setup of every test does the trick:

And the isUITest() method will look like:

But what about if we want to have extra configurations beside mocking the network; for example, mocking the current location? Let’s put all this together.

The next step will be to create a MockAppDelegate that will act as AppDelegate only when we are in a UItest session. The result is as follows:

In order to use the MockAppDelegate only when running tests, let’s create a main.swift file that will use this MockAppDelegate when we are running UI tests. So we’ll need to launch an extra environment in test setup. We called it isRunningTest. The final result of the main.swift is:

Let’s recap what we have at the moment:

1. Before each test execution, we launch a couple of arguments/environments.
2. We use a MockAppDelegate instead of the regular AppDelegate that will behave like the application but will allow us to mock the network.
3. We forward all the network requests to our custom URLSession that we named MockUrlSession()

Finally the time to discover the real magic of our tests has come. What exactly is MockUrlSession? At the end we want to keep it as simple as possible so it’s an URLSession that will override the dataTask.

That will forward all our network requests to our custom MockDataTask and, at the end, our MockDataTask will be:

And getResponseBody() will return the status code of the response and the body.

A Slice Of The Pie

Let’s put our solution into action. Let’s go to the simplest possible test: log into our app. To keep it simple, let’s imagine that the login is a POST operation with the endpoint api.apple.pie/login.

Will it work? Of course not! We are missing something. How do we make it so the login responds to what the app is expecting? We need something else to make it work.

Let’s use our lovely environments to solve it. We are going to launch another environment executing in our setUp method:

bot.app.launchEnvironment["Login"] = "Success"

Now we have to configure what our response will be in case of success. It’ll be done in the method getResponseBody mentioned a few lines below:

If we want to go one step forward and avoid maintaining API responses (one of the requirements mentioned before) the DTOs of our app can be used to build the responses.

As a drawback of this mocking approach, we won’t be able to have more than one response to a given request during the same test execution because the configuration is set as this environment. This cannot be modified after the app is launched (it’s specified in the Apple documentation of launchEnvironment and launchArgument). At the moment, we haven’t faced this inconvenience (it’s a good sign as the tests should only test one behaviour) but we have some workarounds in our mind to deal with this situation.

I hope you enjoyed the recipe! Send a picture with the result in case you to do it at home and don’t hesitate to let us know some special cinnamon that can make the Apple pie even better!

Like what you see? We’re hiring! 🚀 Check out our open engineering positions.