Pipeline Parallel Stages: Make XCUITest Charming Again

We have used XCUITest for iOS automation testing for a long time. As time goes by, we have more and more tests. Of course, it takes more time to complete all the tests.

However, we just consider it as a time-consuming job. In the end, it takes almost three hours to complete all the tests.

Three hours, it’s not a charming number for continuous integration. Since we aim to speed up our development process, get timely feedback from automation testing really matters.

That says, we must find a solution to shorten automation cycle.

Use Xcode scheme to manage tests

Due to the limitation of our product, we can not just execute any test in parallel.

We need to organize our tests, separate tests into proper groups. We use Xcode scheme to achieve it.

As you can see here, we have several testing schemes and assign different test groups to different schemes.

Instead of Xcode scheme, you may use Xcode test plan

Therefore, we can execute test groups with each scheme independently.

In other words, we can perform these tests in parallel.

Pipeline parallel stages

Next, we should adjust the Jenkins configuration.

A simple way to achieve parallel testing is to create Jenkins jobs for each Xcode scheme. Since we use pipeline, there is a better solution, parallel stages.

With following script, we can execute each scheme at different nodes in parallel.

Here’s how it looks like on Jenkins Blue Ocean, we actually create three parallel stages.

After all, it takes ninety minutes to complete automation testing. It’s a huge improvement!

Mostly, we start on a new path of parallel testing for XCUITest. Although we will have more tests in the future, automation cycle won’t extend. We will have more parallel stages to deal with the increasing tests.

Moving forward, we will strive to complete automation testing within an hour, so that we can get timely feedback. It’s quite a challenge but can definitely help us in product developing.