Continuous Testing for iOS Apps using XCUITest, Bitrise & Browserstack
iOS Apps Continuous Testing journey is really exciting for anyone who is trying to implement it using different tools/framework combinations.
In this article, I will share about how it can be implemented using XCUITest, Bitrise and Browserstack platforms.
We used XCUITest automation framework provided by Apple and POM design pattern for writing tests which helped us to write clean and maintainable tests.
Bitrise continuous Integration system enabled us to create pipelines which can build both iOS and android projects without worring about managing build environments/machines.
Browserstack served the need of a Cloud Testing platform well to enable “Continuous Testing” for iOS and android apps.
Let me share few of the tips that will make “Continuous Testing” seamless in your Continuous Delivery pipelines:
- Avoid building “UI Automation Scheme/Tests” in the release pipelines: If you are building your xcuitests as well in the release pipelines (release to beta/appstore), this will increase the build time and might act like additional build step which would take almost same time of building your ipa. The optimized approach would be to build your “XCUITests” scheme on a nightly basis(we created scheduled jobs in Bitrice-CI) and publish the xcuitests to your cloud testing platform. Browserstack has handy apis using which you can publish your xcuitests on the cloud, which can be used for executing the tests later in the release pipelines.
Bitrise-CI: Scheduled Job View:
Now your release pipeline steps will be optimized like below: (ie: once the ipa is built, just publish it to Browserstack and then start executing tests using previously published xcui_test_suites)
2. Group your tests:
We classified our tests into BVT, Smoke, Sanity and Regression and created separated files to store tests details and device to execute the test as shown below:
3. Use the newly available Bitrise step “Browserstack iOS UI Test” to specify test configurations like ipa url, xcuitest url, tests to execute etc. [ Really excited to become the creator/maintainer of this step :) ]
Following are the configuration options available on this step:
As we all know that “UI Automation” is sometimes flaky due to different reasons, automatic re-try for failed tests is a must have feature in your CI pipelines. This step supports automatic retry as well to make ui test execution seemless in the continous testing pipelines.
Also, “Browserstack local testing feature also is supported via this step”. One requirement is, you need to make sure Browserstack local binary is running before executing this step and jq pacakge is installed on the node on which this step is running.
That’s it. Now you should be able to run your XCUITests on Browserstack successfully.
Test Results from Browserstack cloud testing platform:
