Achieving higher test coverage: Our approach to UI testing on iOS

By Aliou Sarr, iOS Developer

Having 28 million unique visitors per month and 800,000 new classified ads posted every day, the leboncoin website and mobile application need to be resilient. With one release every week and 40 developers working across different feature teams, the iOS app is particularly vulnerable to regressions. So, as a team, we asked ourselves: What can we do to ensure that new features do not make the app crash or break? One of the obvious solutions was to improve our testing processes. Therefore, we spent some time optimizing them, particularly our UI tests, and the results are good: In less than a year, the number of UI tests written by developers has doubled, resulting in higher test coverage. So, in this article, we’ll be sharing what we’ve put in place at leboncoin to make writing UI tests easier for iOS applications.

Context

At leboncoin, the tech team is divided into feature teams, with up to three iOS developers in each, depending on the scope of the project. Because we are 40 iOS developers, we had to segment the app into 50 functional and technical modules to avoid conflicts. Among the functional modules are search, payment, and account management, whereas the technical modules include network requests, data storage, and tracking.

Our CI/CD is powered by Zuul and Fastlane and the unit and UI tests are executed by the CI/CD pipeline before each review process. The pipeline then generates an IPA with all the necessary frameworks, builds, and runs integration checks several times a day to make sure our app is bug-free. The app is then submitted to the App Store using these builds.

But enough context, let’s dive into what we’ve done to simplify UI test writing at leboncoin in order to get better test coverage.

Giving developers ownership of UI tests

In order to ensure the quality of our applications, a QA team was created at leboncoin in 2013. They were responsible for writing UI tests for the iOS app, among other things. However, what appeared to be a step forward for us, the iOS developers, soon became a bottleneck.

The old release process: QA engineers were in charge of writing UI tests

Here’s how it worked:

1. The IPA was built on CI/CD.
2. Unit tests were executed.
3. UI tests were run by QA on the IPA.
4. If the UI tests passed, the version was uploaded to the App Store.

There were numerous downsides to the old release process:

• Since the UI tests were written by QA engineers who were not part of the feature teams, there was always a delay between the development of new features and the updating of the tests. Since new features weren’t being taken into account in the tests yet, certain use cases were not covered by those tests.
• When tests failed, production releases were often delayed because the QA team had to update the tests.
• Sometimes there were several days between commits and tests, which made fixing potential regressions more challenging.

That’s why we decided to put iOS developers back in charge of creating UI tests.

The new release process: Developers are now in charge of writing UI tests

As a result, the process has been updated as follows:

UI tests are written by iOS developers when working on a feature.

1. The IPA is built on CI/CD.
2. Unit tests and UI tests are executed.
3. If the UI tests pass, the version is uploaded to the App Store.

This new process has allowed us to take control of UI tests again, which makes sense since we have the most knowledge and expertise about the features. A failed test can now be updated directly without us having to depend on another team.

Using a framework to simplify UI test writing

The process of creating UI tests can be challenging for iOS developers.

XCTest, the native iOS testing framework, can be unstable, depending on the version of the app and parameter settings. And understanding how to test a component or element often requires investigation, despite there being XCTest properties.

In addition, many perturbations can occur on the app (like consent pop-ups) that prevent iOS developers from testing the interface correctly.

Also, UI tests are not standardized, making the process of writing and reading difficult.

Native UI tests

With regard to forms, here’s what native UI tests look like — we need to test the below form to:

• Make sure the validate button is disabled initially.
• Make sure the first email field is filled out properly.
• Ensure the confirmation email field is identical to the first email field.
• Check that the validate button is enabled once all the fields are filled out without errors.

You can see from the above that the UI tests for this basic form are complex enough! That’s why we ended up creating our own framework at leboncoin.

Our in-house framework: TestKit

We designed our internal framework to be simple and easy to maintain. The goal was to get developers to think about what they should test, rather than how they should test it.

Our previous example now looks like this thanks to TestKit:

The code is easier to read: It is much simpler to understand the logic behind the tests, so adjustments can be made more rapidly.

How does it work? TestKit encapsulates XCTest by providing helper methods to simplify UI testing. To do this, all the testable elements available in UIKit (such as Button, TextView, TableView, ActionSheet) are wrapped into helpers.

The following example shows the components that can be handled, along with their wrapper protocols:

Focusing on the Button wrapper, we can see that it contains three methods for testing an element:

• tap(): to simulate a tap gesture.
• validate(isEnable:): to determine whether the button is enabled.
• validate(text:): to check the button’s title.

Now let’s implement the wrapper:

Automatically synchronizing UI test execution status with a QA test plan

Even though iOS developers are now responsible for writing UI tests, it is the responsibility of QA and the feature teams to ensure the iOS app tests are executed. QA engineers at leboncoin were able to link their test plan with the tests run by iOS developers by assigning an ID to each test case. When a test passes or fails, the status is automatically updated in the test plan, making communication between iOS developers and QA easier than ever.

An example of a test plan would be:

// GIVEN

Display account view

// WHEN

Tap on log button

// THEN

Show login form

Developers need to set a reference in their tests to find out which UATs are covered. To do this, they add an attachment to the xcresult.

Once the test runs have been performed in CI/CD, the results are collected and stored by QA engineers in a database. After that, they are able to provide QA analysts with a report.

Enabling UI test execution in different configurations

Wouldn’t it be great if you could run the same UI test in different configurations? If you could test the same feature whether logged in or logged out, for instance, or launch any step of the application directly? It’s now possible at leboncoin! With the help of a protocol, we can get the value associated with the configuration in the app properties.

We use a module that retrieves two parameters that are necessary to launch the test: The name of the rootView to display and the test configuration that is provided by the UI test in the form of a serialized object. This configuration is passed via the environment variables. In the app, this data is retrieved, deserialized, then used to initialize the data and display the correct view controller.

That’s all folks!

The work we’ve done to simplify the UI testing process at leboncoin has undoubtedly had an impact. It has allowed us to achieve a higher test-coverage rate in our iOS app doubling the number of UI tests, resulting in a 99.6% crash-free app. So what’s next? Open-sourcing our framework TestKit as a way to give back to the iOS community. Additionally, we would like to implement snapshot testing to help us test static elements. And there is no doubt that we will face some new technical challenges, such as the management of multiple languages on the app. So stay tuned for our next article!