Speed up iOS CI using Test Without Building, xctestrun and Fastlane

PS: This post originally published on my personal blog ; XCBlog here

At WWDC 2016, there was an awesome talk on ‘Advance Testing and Continuous Integration‘ which mentioned lot of new features in the XCTest Framework, Xcode-Sever and xcodebuild command line tool. We can use those features to speed up the iOS Continuous Integration process.

Current iOS CI Limitation

Before Xcode 8, we have to build and compile and application before running Unit Tests and UI tests on CI which is duplication of running task. The distributed testing is time consuming as well as we have build and compile the source code on every machine. Actually building and Compiling takes lot of build time on CI Server.

Xcode 8 and xcodebuild Features

Xcode 8 release bought couple of new features in ‘xcodebuild’ which can add lot of value to iOS development and testing process.

‘xcodebuild‘ is the command line tool to build, run and execute our application from command line. This is used in the Xcode server. Xcode 8 now has some improvements in the xcodebuild command line tools.

Build For Testing

The xcodebuild now has ‘build-for-testing’ option, it takes workspace scheme and destination as usual but on top of that it will create ‘XCTESTRUN’ file. We just need to execute ‘build-for-testing’ action

$ xcodebuild -workspace <your_xcworkspace> -scheme <your_scheme> -sdk iphonesimulator -destination ‘platform=iOS Simulator,name=<your_simulator>,OS=10.2’ build-for-testing

This should build our app for the testing and create xctestrun file in the DerivedData.

Test without Building

The xcodebuid also has another option called ‘test-without-building’ where we don’t need to provide workspace instead we specify the XCTESTRUN file which will inject that file and runs all the tests.

This feature can be highly useful for the distributed testing as we can created XCTESTRUN file at one machine and distributed to other test specific machine. In order to use it we can specify this option to ru tests without building

$ xcodebuild -workspace <your_xcworkspace> -scheme <your_scheme> -sdk iphonesimulator -destination ‘platform=iOS Simulator,name=<your_simulator>,OS=10.2’ test-without-building

This time it won’t build and application.

Only-Testing/Skip-Testing

Suppose, you don’t want to run your unit test. Xcodebuild now has two new testing options

• –only-testing : Include Test suites
• –skip-testing : Exclude Test suits

$ xcodebuild -workspace <your_xcworkspace> -scheme <your_scheme> -sdk iphonesimulator -destination ‘platform=iOS Simulator,name=<your_simulator>,OS=10.2’ test-without-building -only-testing:<your_test_bundle_to_run>

Speeding up the XCTest on CI

Using the features mentioned above, we can speed up the test execution as well as save the build time. Let’s jump into the Xcode and see how to do that

Setup Xcode Schemes

Create a sample iOS app in the Xcode and name it XCTestRun with unit and UI tests included. You may also want to save this project as ‘workspace’ as most of the real projects are saved as ‘.workspace’. You can do that using File->Save as Workspace Xcode option. You can also create separate schemes to run Unit Tests and UI Tests. Thats it

Build For Testing

Let’s now try to build this project using new features ofxcodebuild . Let’s first use build-for-testing feature to prepare our application for testing.

$ xcodebuild build-for-testing -workspace “XCTestRun.xcworkspace” -scheme “XCTestRun” -destination “platform=iOS Simulator,name=iPhone 7,OS=10.2” -derivedDataPath “build”

This will build an application for testing and creates XCTestRun_iphonesimulator10.2-x86_64.xctestrun file inside the build/Build/Products directory. We can now use this file to run tests without building.

Test Without Building using xctestrun

We can now use this file to run tests without building using test-without-building options, we can also use -only-testing and — skip-testing

$ xcodebuild test-without-building -xctestrun “build/Build/Products/XCTestRun_iphonesimulator10.2-x86_64.xctestrun” -destination “platform=iOS Simulator,name=iPhone 7,OS=10.2” -only-testing:XCTestRunTests/XCTestRunTests.swift

We can execute those script as part of CI or we can use tools like Fastlane mentioned below.

Using Fastlane

Fastlane has set of tools to use for iOS automation as well as loads of actions to perform tasks. In order to setup Fastlane we need to create Fastlane directory and Fastfile inside that directory.

Fastlane has xcodebuild action as well scan tool but unfortunately none of support build-for-testing and test-without-building natively without additional effort at the time writing this post. We need to write custom action to perform those steps. I have written custom Fastlane Action called ‘xctestrun‘ which takes few options to run the build and test without building.

Now that, we have custom action, let’s us that in the Fastfile by adding the following code.

require_relative ‘actions/xctestrun’
require_relative ‘xcode8-xctestrun’
Xcode8.runner = self.runner
fastlane_version “2.19.3”
default_platform :ios
desc “Test witout building”
lane :build_for_testing do
xctestrun(xcodebuild_action: ‘build-for-testing’, scheme:   “XCTestRun”, workspace: “XCTestRun.xcworkspace”, destination: “platform=iOS Simulator,name=iPhone 7,OS=10.2”)
end
lane :unittest do
xctestrun(xcodebuild_action: ‘test-without-building’, scheme: “unittest”, workspace: “XCTestRun.xcworkspace”, destination: “platform=iOS Simulator,name=iPhone 7,OS=10.2”)
end
lane :uitest do
  xctestrun(xcodebuild_action: ‘test-without-building’, scheme:   “uitest”, workspace: “XCTestRun.xcworkspace”, destination: “platform=iOS Simulator,name=iPhone 7,OS=10.2”)
end

This fast file has separate lanes for ‘build-for-testing’ , running Unit tests and UI tests also note that we are using custom action ‘xctestrun’

We can now run fastlane lane ‘build-for testing’ to build and app for testing.

$ fastlane build_for_testing

This will build an app for testing

Now, our app has been built for testing, let’s execute Unit tests using ‘unit test’ lane

$ fastlane unittest

This will execute only ‘unittest’ scheme.

Similarly, we can execute UI tests

$ fastlane uitest

This will execute only UI Tests.

We don’t need to build or compile an app to run unit and UI tests.

Source Code

The Source code for this demo is available on Github ‘XCTestRun‘ You can try it yourself.

Conclusion

Using new features of ‘xcodebuild’ we can save build time of iOS Continuous Integration process and achieve distributed testing by passing xctestrun file to multiple test machines. This will definitely speed up iOS Continuous Integration Process.

Like this post from XCBlog By XCTEQ ? You may also like some of our services like guest blogging or Mobile DevOps(CI/CD) or Test Automation. Chekout our services, open source projects on Github or Follow us on Twitter , Facebook, Youtube , LinkedIn. Download Our XCBlog iOS App to read the blogs offline.

XCTEQ Limited: Mobile DevOps, CI/CD and Automation

XCTEQ is a company specialised in Mobile DevOps, CI/CD, Mobile, AI/ML based Test Automation Checkout XCTEQ products and services at http://www.xcteq.co.uk or write to us on info@xcteq.co.uk..