Hands-on XCUITest Features with Xcode 9

Note: This post is originally published on my personal blog here. Continue reading there as some of the GIFs aren’t uploaded on Medium.

At WWDC 2017, there was a great session on What’s New in Testing which was mainly about new features of XCTest and XCUITest frameworks. The team working on developer tools at Apple has made huge improvements in the area of UI Testing with XCUITest and Continuous Integration with Xcode Server. In this post, we will explore all the new features with practical examples in Xcode 9 and command line. There is Github repo Xcode9-XCTest created as part of this exploration. You can reference this post to that GitHub repository to try out the things by your own.

There are lot of new things announced related to testing for Apple platforms especially iOS and macOS. Some of them are as below.

• XCUISiriService
• Localisation Testing
• Async Testing
• UI Test Performance Improvement
• Activities, Attachments and Screenshots
• Multi-App Testing
• xcodebuild: Headless Testing
• xcodebuild: Parallel Testing
• Inbuilt Xcode Server

There is lot of enhancement in testing processes including Siri integration, Waiting in XCTest, Core Simulators in xcodebuild and many more. Let’s dive into these one by one.

XCUISiriService

Using XCUISiriService, we can now interact with Siri interface. We can control Siri by passing voice recognition text from our XCUITests and Siri will act accordingly. Let’s imagine, we want to open an Apple News app using XCUISiriService, we can do that from our test.

func testXCUISiriService() {
        XCUIDevice().siriService.activate(voiceRecognitionText: "Open News")
    }

I wrote a detailed blog about Controlling Siri from XCTest using XCUISiriService few months ago when it’s announced with Xcode 8.3 also created demo project on Github XCUISiriServiceDemo but API syntax have changed a bit but examples are still valid.

Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testXCUISiriService() test from Xcode9_XCTestUITests.swift file. You can see the Siri will open Apple News app.

Localisation

With Xcode 9 , we can use Xcode scheme to run test in different Language and Region. When we edit the scheme, we see the Test options to run tests using specific languages and Regions. We can easily test our app with different language and regions by changing the scheme settings.

Async Testing

There are many situations where we need to wait till things to happen to carry on our test execution like opening document, waiting for response from server etc etc but it’s most common in UI testing scenarios. As of now, XCTest handles async testing by creating expectations and test will wait till the expectaion fulfilled. The XCTest timeout will result in test failure. The expectations are tightly coupled with XCTestCase.

Now, XCTest has new class XCTWaiter which allows us to define expectations explicitly which is decoupled from XCTestCase. It has initialiser as public API and then different waiting conditions covered. XCTWaiter waits for group of expectations to be fulfilled. We can now define the expectations like this:

wait(for: [documentExpectation], timeout: 10)
 // OR
XCTWaiter(delegate.self).wait(for: [documentExpectation], timeout: 10)
// OR
let result = XCTWaiter.wait(for: [documentExpectation], timeout: 10)
if result ==.timeout {
   ...
}

The list of the newly added classes and sub-classes are as follows:

• XCTWaiter : Takes control over to the waiting strategy.
• XCTNSPredicateExpectation: Readable expectation while writing automate tests
• XCTKVOExpectation: Readable expectation while writing automate tests
• XCTDarwinNotificationExpectation: Another readable expectation for notifications.

There is one more handy API that we can use to wait for existance of XCUIElement.

XCUIElement.waitForExistance()

Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testXCWaiterfeatures() test from Xcode9_XCTestUITests.swift file. I have already shared detailed article few months ago on Asynchronous iOS Testing in Swift with XCWaiter also there is demo project on GitHub here

UI Testing Performance Improvements

Removal of Snapshot

As we are aware of the fact that, XCUIApplication is a proxy app that interact with main app which is used to launch, terminate and query the main application. Apple was using snapshot tool to communicate between XCUI app and main App which was causing performance issues in terms of time and memory. Now snapshot has been removed and replaced with remote query and query analysis technique to improve performance of the XCUItests. We might have faster execution of the tests.

First Match API

Apple also introduced First Match API to speed up response for the query. First Match API will exit early as soon as it finds the first XCUIElement rather than querying everything. We can use firstMatch on any XCUIElement

let button = app.navigationBars.buttons["Done"].firstMatch

Now the query would have faster magnitude and no memory spike.

Match First Vs Match All

app.buttons.firstMatch //Not Good idea
app.buttons["Done"].firstMatch // Better
app.navigationsBars.buttons["Done"].firstMatch //best

Activities, Attachments and Screenshots

There are big improvements in the area of organising tests for readable reports, taking screenshots of XCUIElements and attaching rich data to test reports. There are three different concepts introduced this year which are

• Activity : Grouping of actions by giving meaningful name
• Screenshot : taking Screenshot of fullscreen or specific XCUIElement
• Attachments : Attaching rich data to XCTest reports.

Activities

UI Tests are usually long running and lot of actions happening there e.g tapping buttons, swiping etc. As of now, XCTest reports shows all the actions in the test reports which is not particularly readable. Activities is the way to organise those actions into group by giving meaningful name so XCTest result will use that activity name in the result to make it more readable. You can read more about activities on Apple official documentations here
We can sprinkle activities on any set of actions e.g launch app in clean state

XCTContext.runActivity(named: "Given I have launched app in clean state") { _ in
   XCUIApplication().launch()
   XCUIApplication().launchArguments = ["-StartFromCleanState", "YES"]
}

Now that, we have defined activity to launch an app in the clean state. When we run the test then in the test report we will see “Given I have launched app in clean state” which is more readable. We can still access underlying actions by expanding the activity.

ScreenShots

Apple also announced the new API to take screenshot of full screen as well specific XCUIElements. There is a new protocol XCUIScreenshotProviding to provide screenshot of current UI state and there are two new classes XCUIScreen and XCUIScreenshot to capture screenshot of app or UIElement state. We can take screenshot using following code snippet.

let screen = XCUIScreen.main
let fullscreenshot = screen.screenshot()

Attachment

An Attachment can be used to attach rich data to test reports. It may be data from test, improved triage additional log, post processing workflow. The data may be in the form of raw binary data, string, property list, code object, files or images. We can attach the screenshots to activities using attachment. We can add attachment to activity like this:

XCTContext.runActivity(named: "When I add attachment to activity") { (activity) in
            let screen = XCUIScreen.main
            let fullscreenshot = screen.screenshot()
            let fullScreenshotAttachment = XCTAttachment(screenshot: fullscreenshot)
            fullScreenshotAttachment.lifetime = .keepAlways
            activity.add(fullScreenshotAttachment)
}

Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testActivitiesScreenShotsAttachments() test from Xcode9_XCTestUITests.swift file. You can see the test reports in Xcode are taking activity name rater than actions. In other test, we have attached screenshots to the activity.

Watch Animated GIF here

Multi App Testing

Previously we can test only one application which is target application for the XCUITest target. There was no way to interact with other apps like Settings or News from XCUITest. XCUIApplication() is used to launch, terminate and query an application under test. It can only access target application under test and doesn’t test other applications. There were always need to access other applications like Settings or App extensions.

With Xcode 9, we can test multiple application using XCUIApplication. It has three main changes, now XCUIApplication()

• has initialiser which takes bundleIdentifier so that we can pass bundleID of our app
• has new activate() method to activate app from background
• has States to monitor changes in application. States has cases like runningBackground , runningForeground etc etc

We can test two applications in one XCTest using below code

func testTwoApps() {
  let app1 = XCUIApplication(bundleIdentifier: "com.app1.xyz ")
  let app2 = XCUIApplication(bundleIdentifier: "com.app2.xyz")
  // Launch App1
    app1.launch()
  // Launch and test App 2
   app2.launch()
   app2.launchArguments = ["-StartFromCleanState", "YES"]
// test App1 again by activating it from back ground
   app1.activate()
}

The activate() method automatically wait for another app to become active but there might be situations where, we want to manually wait for app to be activated with can done by using predicates and XCUIApplication states.

let app1ActiveExpectation = XCTNSPredicateExpectation(predicate: NSPredicate(format: "state == \(XCUIApplication.State.runingForeground.rawValue"), object: app1)
wait(for: [app1ActiveExpectation], timeout: 10)

As we can see that, we can interact with any app within Simulator or device as long as we know bundle identifier. This is huge improvement in UI testing of iOS apps.

Practical Example

Open the Xcode9-XCTest project in Xcode 9 and run the testMultipleApps() test from Xcode9_XCTestUITests.swift file. We are interacting with main application and settings app.

Note: Currently test is failing while launching settings up but simulator is launching settings app regardless.Watch GIF here

xcodebuild : Headless Testing

There are some huge improvements in the xcodebuild command line tool which is used for analysing, building, testing and archiving an iOS application. xcodebuild will use core simulator to run tests so we don’t see simulators running when running our tests from command line. It means we won’t see simulators on CI servers. It will be totally headless.

Practical Example

Clone or download Xcode9-XCTest project and run xcodebuild command. One thing to note that , there is no simulator launching while tests are running.

$ git clone https://github.com/Shashikant86/Xcode9-XCTest
$ cd Xcode9-XCTest
$ xcodebuild -scheme "Xcode9-XCTest" -destination 'platform=iOS Simulator,name=iPhone 7 Plus,OS=11.0' build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO

xcodebuild: Parallel Testing

xcodebuild support parallel device testing. It means we can run test in multiple simulator and devices as long as devices are provisioned also we can run test on devices which are wirelessly connected to server. We just need to pass multiple destinations to xcodebuild.

Practical Example

Same as above Clone or download Xcode9-XCTest project and run xcodebuild command by passing additional destinations options.

$ xcodebuild -scheme "Xcode9-XCTest" -destination 'platform=iOS Simulator,name=iPhone 7 Plus,OS=11.0'-destination 'platform=iOS Simulator,name=iPhone 7 Plus,OS=10.3' build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO

Inbuilt Xcode Server

Xcode Server is a continuous integration system provided by Apple. As of now Xcode Server need macOS Server application to run Xcode bots. Now, Xcode Server no longer need macOS Server application. It’s inbuilt in Xcode and can be accessed from Xcode Preference. Xcode Server has improved provisoning, uses core simulator (headless) for running test , parallel testing multiple device , Localisation control.

Practical Example

I have shared a detailed article about how to setup Xcode Server using Xcode 9 which is Xcode 9 + Xcode Server = Comprehensive iOS Continuous Integration. Feel free to navigate to that article to learn more about Xcode Server setup from scratch.

Watch GIF here how to create Xcode bot for our demo project Xcode9-XCTest

Conclusion

New features in XCTest framework like parallel testing, automatic provisioning, multi app testing will definitely be useful for all the iOS teams across the world. Apple is investing time on improving testing and continuous integration practices. Hope it will continue in recent years.

Like this post from XCBlog By XCTEQ ? You may also like some of our services like guest blogging or Mobile DevOps(CI/CD) or Test Automation. Chekout our services, open source projects on Github or Follow us on Twitter , Facebook, Youtube , LinkedIn. Download Our XCBlog iOS App to read the blogs offline.

XCTEQ Limited: Mobile DevOps, CI/CD and Automation

XCTEQ is a company specialised in Mobile DevOps, CI/CD, Mobile, AI/ML based Test Automation Checkout XCTEQ products and services at http://www.xcteq.co.uk or write to us on info@xcteq.co.uk..