Automated StoreKit Testing at Wattpad
If you have worked with in-app purchases before, you likely know that purchasing an IAP is not supported on the iOS simulator. Is it possible to automate tests with in-app purchases? The answer is yes, this post will demonstrate how we do so at Wattpad. Before we get into the how, it is important to understand why investing in testing matters in the first place.
Our Testing Culture
At Wattpad, we have a strong testing culture. For Wattpad’s iOS app, we write unit tests for any change in business logic. When it comes to UI changes, we have a couple of options. Depending on the case, we could write snapshot tests or automated UI tests. We follow the testing hierarchy to determine which tests to write. For example, we have snapshot tests for one of the UICollectionView cells which represents a story to ensure it plays well with long titles and localized strings. On the other hand, we have the reading experience which is one of the core features of the app. Not only do we have unit tests for the reader, we also have several UI tests for the most important user flows in the reader.
We recently added subscriptions and in-app purchases to our app to enhance the user experience and support the writers on our platform. Needless to say, it is critical that the purchasing user flows work as expected. We have extensive unit test coverage for them. However, UI tests that cover purchasing a subscription or a pack of coins would be the cherry on top. So, we set out to do that.
Anatomy of a Test
We wanted to write UI tests that mimic a real-world user, and a real-world user would have to go through several StoreKit prompts in order to make a purchase. Additionally, our tests should be deterministic. In other words, no matter how many times a test is run, it should run in the same environment and conditions leading to the same result. So all of our tests follow this structure:
Setup
All of our tests leverage iOS 12 “sign in with sandbox account” feature. At the start of each test we go into device settings and sign out of an existing sandbox account if needed. One important trick we use for consistent results is to open the Settings app, then kill it, then open it again to reset the state. Even though Apple’s documentation for XCUIApplication.launch() states it terminates the app before launching, we’ve found this trick to be more reliable.
To launch settings:
Navigate to Sandbox Account in Settings and sign out:
You’ll notice in the code samples above we use some convenience methods waitForIt() and waitToBeHittable(), they can be found here:
Purchase and Verification
Once all the normal UI test stuff of navigating to the screen you want to test is complete, we reach the meat and potatoes of a test. We need to log into a sandbox account, make the purchase, then recognize and handle any dialogues that pop up.
Tap on an IAP to trigger the sign in flow
There are two cases when tapping the IAP, you can get either the `App Store and Privacy` alert or the `Sign in Required` alert, we need to handle both.
First of all, how do we recognize the alert?
let springboardApp = XCUIApplication("com.apple.springboard")
let privacyAlert = springboardApp.alerts["App Store & Privacy"]
if privacyAlert.waitForExistence(timeout: 30) { ... }Handle the alert
let alert = springboardApp.alerts["Sign-In Required"].waitForIt()let emailTextField = alert.textFields["Apple ID"].waitForIt().waitToBeHittable()emailTextField.tap()emailTextField.typeText(account.email)let passwordTextField = alert.secureTextFields["Password"].waitForIt().waitToBeHittable()passwordTextField.tap()passwordTextField.typeText(account.password)alert.buttons["Buy"].waitForIt().waitToBeHittable().tap()
Once the purchase goes through…
let subTermsAlert = springboardApp.alerts["Subscription Terms"].waitForIt()
subTermsAlert.buttons["Continue"].waitForIt().waitToBeHittable().tap()
let confirmSubAlert = springboardApp.alerts["Confirm Subscription"].waitForIt()
confirmSubAlert.buttons["OK"].waitForIt().waitToBeHittable().tap()However, sometimes the purchase can fail and we can be met with any number of failures. Luckily we learned from the example above how to recognize and handle a springboard alert. A common failure we run into with testing subscriptions is `You are not authorized to make purchases of this InApp in Sandbox at this time.` In order to handle this we need to retry the purchase from a clean state, so we switch to the Settings app, sign out of the sandbox account and retry. Note that we use a lot of helper functions, so copy pasting these code examples may not be appropriate. Sharing understanding of the roadblocks and how to navigate by them is what we hope to share from these examples.
Finally, we reach it. The alert that lets you breathe a sigh of relief when you tested your first subscription or IAP. The 3 magical words. You’re all set. This means the hard part is over and we can verify our states, make our assertions, and other testing stuff we all know.
Next Steps
Extracting the steps that interact with the StoreKit prompts allowed us to write new tests involving purchases with ease. We are still working on addressing a couple of ongoing issues with these tests: they take a long time to run as we wait for possible prompts and there is occasional instability with unknown purchase errors. Aside from these issues, we now have a growing bank of purchase tests that run on the cloud using AWS device farm.
We hope that sharing our journey through purchase testing was helpful.
How does your team do purchase testing? Let us know!
