Stubbing Apple’s UI Tests
A couple of months ago we decided to revisit the automatic user interface test suites of our iOS app trying to approach the subject with a fresh set of mind. The existing code base we had consisted on a fairly large number of test cases that used the popular KIF testing framework which, at the time we adopted it, was de facto the only available option. The tool is great but to be honest we experienced some problems with it, the biggest being that we struggled to achieve consistent results. There was always one test randomly failing which, over time, made us trust the test results less and less.
Ever tried UI Tests?
We decided to start from scratch using Apple’s latest interface testing technology, UI Tests. This framework comes with all the benefits of being an official tool: it’s well integrated with the development tools, it will be always up to date and last but not least one would expect it should just work. And indeed, a part from a rough start, it does not just work but it’s rock solid.
At first glance it seems that Apple’s framework offers similar functionalities to KIF, however there is one significant difference with how tests are executed. Testing code is not embedded inside your application, instead it’s part of a separate app that is installed on the device and is in charge to execute the user interface automation. If you google around you’ll find that this is seen as being the major flaw in UI Test because test code and app code live in different processes, preventing the testing code to access the app’s code. This means that there’s no way to stub network requests with OHHTPStubs as we were used to.
When that’s not enough
Before moving on adopting UI Tests we had to restore some of the basic functionalities we were used with KIF, most importantly network request stubbing. The solution we found was pretty simple: an http server is instantiated inside the app receiving requests from the testing code. This allows to dynamically send requests during the execution of the tests, giving great flexibility in how tests are designed. The result is a library, SBTUITestTunnel, that wraps all the complexities of its implementation and is easy to use and integrate in any project.
With the library in place, in your test target, you’ll be able to easily:
- stub network calls
- interact with NSUserDefaults and Keychain
- download/upload files from/to the app’s sandbox
- monitor network calls
- define custom blocks of codes executed in the application target
Really? (for TL;DRers: Installation)
Integrating SBTUITestTunnel with CocoaPods is the most simple way to get you started. You’ll need to add two pods to the Podfile, one for the UI Test (XCTestsCase subclasses and other boiler code) and the other for the application’s target (the http server that receives and handles the requests).
🔥🔥🔥 If you’re using CocoaPods v1.0 and your UI Tests fail to start, you may need to add $(FRAMEWORK_SEARCH_PATHS) to your Runpath Search Paths in the Build Settings of the UI Test target!
Then modify your AppDelegate.swift importing the SBTUITestTunnel framework and let the takeOff() method do the rest. It should look something like this:
🔥 The entire code of the library is wrapped around DEBUG preprocessor macros so that SBTUITestTunnel won’t be included in your production code!
❓ Why the initialize() method? Launching SBTUITestTunnel there allows us to synchronously execute code a tiny bit before the app starts, allowing for example to conveniently set user preferences.
That’s it, now you’re ready to write the first test.
Show me the code! (for TL;DRers: Setup)
On the test target you’ll be using our XCUIApplication sublass SBTUIApplication which has the methods to leverage the extra features we added.
When launching the test target you can provide options to perform certain actions at startup and a block which will be executed right before the app starts:
- SBTUITunneledApplicationLaunchOptionResetFilesystem resets the entire app sandbox before start.
- SBTUITunneledApplicationLaunchOptionDisableUITextFieldAutocomplete will disable UITextFields autocomplete feature that often interferes when text is typed in.
- The startup block is the right place to prepare the app status with additional setups (inject files, modify NSUserDefaults, etc).
👀 no more XCUIApplication here and we’re using launchTunnelWithOptions() instead of the usual launch() method
Still having doubts? SBTUITestTunnel headers are well commented making the library’s functionality self explanatory, so make sure to check them out. Alternatively you can also take a look at the example project that we’re hosting on our GitHub repo.
Stubbing (for TL;DRers: Stubbing)
You can stub network calls, conveniently faking the response, using a regex matching a certain request or by checking the queries containing certain parameters. You can optionally remove the stub during execution, although this is not necessary if you need the stub until the end of the test.
Preferences (for TL;DRers: NSUserDefaults and Keychain)
Modifying NSUserDefaults / Keychain is easy. You could for example write a test that verifies that a counter you’re using to track the app’s usage gets properly updated. Or you could “login” a test user by conveniently writing the app’s keychain.
Filesystem (for TL;DRers: Filesystem)
If needed you can upload or download files from the test bundle to the apps bundle. This might come in handy.
Monitoring (for TL;DRers: Monitoring)
Often you need to verify that certain network calls are made after interacting with the UI. Similarly to stubbing the library offers a simple way to do so.
Last but not least (for TL;DRers: custom block of code)
The set of commands offered by the library should widely cover the testing needs of most of us. There’s however always an edge case that needs special treatment. The library allows you to invoke from your test’s target (in the example below CustomBlockTest.swift) a block of code in the application’s target (in the example below ViewController.swift). We advise using this as a last resort because you’ll be going to pollute your application target with test code, but it’s good to know it’s there.
Let’s face it: testing is no easy matter. It can get tricky but once you get it right it’s an irreplaceable tool for a happy development team! We’re enjoying and are proud of how our UI Testing are shaping up and hope that SBTUITestTunnel will help you too.
This post was originally posted on https://tech.subito.it/2016/07/19/stubbing-apples-ui-tests/