How to setup Bitrise CI to run automated Appium and Cucumber UI tests for iOS
Why this stack?
In codequest we’ve been using Bitrise as a CI/CD tool for our iOS apps for a while. Recently, our team has grown and we’ve gained a new most valued member -QA Engineer, who’s got knowledge of writing UI tests with Appium and Cucumber.rb (Ruby version). The question was - how to set it all up in Bitrise, where currently we have only automated unit tests running and testing app deployment.
Probably most of you reading this are looking for some help, just like we were. And to our astonishment - even googling couldn’t help us find a step by step instruction. So I won’t go into many details and cut to the chase.
Our environment
A few details that will be needed in the next steps. Let’s say that:
- Unicorn - this is our project name
- Unicorn.xcworkspace - workspace of our project (we use Pods)
- Unicorn - Xcode scheme name to build
- Unicorn.app - name of the binary created by the build, can be checked in Xcode ➞ (chosen target) ➞ Build Settings ➞ Product Name
- com.codequest.Unicorn - our app bundle id
- AppiumUITests/Unicorn - under this path in git repository we keep the UI tests
- We don’t keep any .app files in the git repository
Create UI tests steps in Bitrise
1. Create a new workflow by clicking on +Workflow button and setting a name.
2. Add “setup” steps by choosing a + button on the right and choose from a window on the right:
- Activate SSH key (RSA private key) — for access to your git repository
- Git Clone Repository
- (optional) Run CocoaPods install — add this step if you use CocoaPods in your project
At the end you should have a workflow similar to this:
3. Add next step called Ruby Script runner with optional Gemfile support.
4. Clear everything in the Ruby script section and exchange Gemfile’s content section with these lines:
What we’re doing here is installing gems needed for Appium and Cucumber to run. The version numbers (e.g. ‘~> 3.1’) can be different when you are using this tutorial. Please check what version your QA uses while writing and running tests on his local machine and have the same on Bitrise.
In case you are wondering - yes, each time a build is started a new environment is set up for Appium and Cucumber.
5. Add a step called Script.
6. Exchange Script content with this script:
What is going on here:
set -exThis short command causes whole script to fail and log the output in case any of the following commands fail.
npm install -g appium
appium &>/dev/null &Installs Appium. Then in the second line - runs the Appium server in a detached process silenced.
xcodebuild -workspace Unicorn.xcworkspace -scheme Unicorn -configuration Release clean build -sdk iphonesimulator CONFIGURATION_BUILD_DIR=$BITRISE_SOURCE_DIR/AppiumUITests/Unicorn/features/support | xcprettyRuns xcodebuild, which means it builds the app that Cucumber will use for testing.
Here we set the workspace name, the scheme to build, configuration and sdk, which has to be a simulator for the UI tests.
CONFIGURATION_BUILD_DIR is setting a path where the build’s .app file will be put. We put it in the directory where Cucumber usually runs the app.
xcpretty makes logs shown on the Bitrise log console, well, prettier.
cucumber -r $BITRISE_SOURCE_DIR/AppiumUITests/Unicorn $BITRISE_SOURCE_DIR/AppiumUITests/UnicornAt last — we run the Cucumber at appropriate path.
7. Optionally you can name the last Script phase in a more meaningful way, so that you won’t forget what the fuss is all about.
Setup in Cucumber env.rb file
Your environment setup file won’t need any adjusting for running UI tests on Bitrise. We copy the .app executable to an appropriate path in a build step on Bitrise, so everything here stays as it is in your local environment.
Your environment setup file should look similar to this, some settings are optional, but your QA Engineer should know how to set it up. For Bitrise the important part is the app settings which need to have your .app name used both locally and in the Bitrise build.
How to combine UI tests steps with your current workflow?
Adding UI tests to our existing workflow wasn’t as easy as it sounded like. We’ve already had unit tests run on Bitrise. By unit tests I mean XCTests from your Xcode (or QuickSpecs, because we use Quick and Nimble for tests).
Here what we’ve learned:
- You can’t use the same app binary for UI and unit testing — unit tests need a Debug version, while UI tests need a Release version, so you have to build your app twice during the workflow. At first we just wanted to create one binary and use it to both UI and unit testing, so the build time is smaller. But it doesn’t work.
- We advise running UI tests before unit tests — to this conclusion we came by trial and error. Although there is a clean build for each binary (Debug for unit tests and Release for UI tests) the simulator in Bitrise had some derived data (or other garbage) not cleaned properly after Debug version was built first and failed to build the Release app. The whole workflow was failing. So we just changed the order and everything started to run smoothly.
Our final steps in a “building” workflow (without app deployment) look like this:
If you don’t like using UI - use bitrise.yml
At last, if you don’t want to click through everything, you can useui_tests workflow’s settings in the bitrise.yml file. You can copy and paste it into your bitrise.yml:
Finally…
I hope this tutorial helps someone. It doesn’t look as much, but it did take as a few days to set it all up (waiting for a build on CI is a pain in the neck). Enjoy!
