Parallel iOS Testing using Bluepill, Fastlane and Jenkins
How to integrate Bluepill using Fastlane and Jenkins
Bluepill is an open source repo that allows for parallel testing of Xcode projects. (https://github.com/linkedin/bluepill) This parallelization reduces test time by observing the runtime complexity of your tests and bundling them in an even distribution of the number of simulators that you choose to run. I will try to walk you through installation of Bluepill and its implementation for Jenkins and Fastlane.
First download the latest release found https://github.com/linkedin/bluepill/releases/ (at the time of this writing it is version 1.1.2). Place the BP and bluepill files inside of your Fastlane folder. Now the repo lists two options for building your test environment, you can either create a bash script or configure a json file. I personally was having trouble getting the json file to run correctly so I made a test script called bluepill.sh, you can modify the contents of this script as needed for your use. (https://gist.github.com/EricADockery/90f806e86113b6d5e29c101a6e665d2a#file-bluepill-sh)
What this script does is back out of our Fastlane folder in the repo grab the derived path of our test build, go back into Fastlane and runs bluepill on 4 simulators retrying the tests once and storing the output in a test_output directory. Then if if finds a failure in the test_output Final report file it exits with a 1 reporting the failure to Jenkins.
Now inside of your fastfile add your bluepill lane:(https://gist.github.com/EricADockery/90f806e86113b6d5e29c101a6e665d2a#file-fastfile)
And lastly update your Fastlane test in Jenkins to run
fastlane bluepill
And watch your tests run in parallel. Determining the right configuration took a couple of days but the results were a reduction in test time by about 15%. I hope this helps you setup bluepill and reduce your test time. One thing that I did notice was that any tests that involve the keychain in our project failed. I am not sure if it is something on our repo or something with bluepill.
Find a bug, or think I’m wrong? Leave me a message and I will take a look.