iOS Automation Testing: Running tests in parallel

5 min readMay 19, 2018

How to reduce test execution time by running in parallel on physical devices and simulators? How to generate test reports with screenshots and run tests via command line interface? Read this article to find out.

Introduction to Xcode UI Testing

This article is prolongation of my article from 6 of May , where we have learned how to setup XCUI Framework for your Project and start writing Xcode UI Tests with Swift. Check it out to get more familiar with the topic.

Getting started with Fastlane Tools

Fastlane are powerful open source tools that will help us to to execute tests via command line interface as well provide us with set of testing, building , signing and deployment libraries. To get started we need to install them:

Open terminal
Execute [sudo] gem install fastlane -NV
Navigate to your project directory and run fastlane init
Type 4 to select manual setup
Wait for setup completion

Now we have Fastlane tools installed. You should have fastlane directory in your project with Appfile and Fastfile created.

Executing tests via command line

In order to run tests on Continuous Integration Server we need to execute it via command line interface. There are multiple approaches for this task, in this tutorial we’ll be using Fastlane Scan action. Let’s set it up:

Navigate to created fastlane directory cd fastlane/
Open Fastfile in text editor (e.g. Atom)
Set scheme according to your project scheme name
Specify test device

lane :test do
    scan(
      scheme: 'BoardBank', # Project scheme name
      clean: true, # Clean project folder before test execution
      device: 'iPhone 8' # Simulator for testing 
    )
end

Congratulations, we have successfully configured our project to execute tests and Fastlane tools will do the rest for us. To run tests:

Execute fastlane test
Wait for tests to be executed

You will find HTML and Junit reports generated in fastlane/test_output directory.

Improving Test Reports

Fastlane allows to generate only simple test reports which are not including screenshots and device logs. When building Test Automation framework we need make sure, that test report has enough information, so we don’t have to re-run tests manually and can analyze tests failures more efficient. Better approach is to use this open source tool. To set up:

brew install TitouanVanBelle/XCTestHTMLReport/xchtmlreport

When we have it successfully installed the next step will be to add a helper method, so we can generate report as a part of test:

def generate_report
    puts "Generating Test Report ..."
    sh 'xchtmlreport -r test_output/BoardBank.test_result'
    puts "Test Report Successfully generated"
end

Running tests in parallel on multiple devices

When testing Mobile Application we need to verify that Application works as expected on all supported devices and OS versions. Fortunately, we can automate this part with minimal effort. Parallel testing on simulators can be configured in Fastfile, we just need to specify devices for testing

TEST_SIMULATORS = ['iPhone 8','iPad Air','iPhone SE','iPhone X']lane :test do
    scan(
      scheme: 'BoardBank', # Project scheme name
      clean: true, # clean project folder before test execution
      devices: TEST_SIMULATORS 
    ) 
end

Unfortunately testing on simulators won’t be always a best approach, cause some issues won’t be appearing. Testing on real devices will provide us more accurate test results. To prepare devices for test execution we need to enable UI Automation on device:

Connect test device to machine
Open Xcode
Run your Application on the device
Open Settings on the device
Search for the Developer section
Enable UI Automation

Also, we need to disable Auto-Lock , so the device will be always ready for test execution:

Navigate to Display & Brightness section
Set Auto-Lock to Never

To make sure, that system notification won’t affect test execution we need to enable Don’t disturb mode :

Navigate to Do Not Disturb section
Enable Do Not Disturb mode
Set Silence to always

The device is configured for test execution now. To execute tests on real device we need to specify UDID. You can get it by multiple ways, for example this. Then we just need to set the physical device as destination for testing:

TEST_DEVICES = [
    'platform=iOS,id=9a31ef5bf7216fa79ec31af3f29901c209175937',
    'platform=iOS,id=8a91ef5bf2036fa49ec31af3f19902c209076390'
  ]private_lane :parrallel_device_test do
    scan(
      scheme: 'BoardBank', # Project scheme name
      clean: true, # Clean project folder before test execution
      destination: TEST_DEVICES, # Devices for testing 
      result_bundle: "TestResults" # To generate test reports
    )
    generate_report 
  end

Tests will be executed on configured devices and test report will be captured for each device. To make sure, that we always generate test reports, we need to write exception handling for the case when testing has failed:

lane :device_test do
    begin
      parrallel_device_test
    rescue
      generate_report
    end
end

Generated HTML report will have test results for each device, as well as test logs with screenshots.

Conclusion

In this tutorial we have learned how to setup Fastlane for your iOS project and execute Xcode UI Tests in parallel on physical devices and simulators.

It will allow to setup our tests on Continuous Integration server and attach advanced test reports, as well as increasing of test coverage and reducing manual compatibility testing effort. Complete Test Framework setup example with Application Source Code is available on GitHub, I highly recommend checking it out to learn more details in context.

Stay tuned for more tutorials and Happy Testing :)

About the Author

Maksim Akifev is a highly skilled Quality Assurance Engineer with wide experience in Test Automation, Security and Penetration Testing who helps Companies in various industries such as Finance, Healthcare, and Cyber Security to build reliable and secure Products.