Self-hosted CI for iOS/Mac development
If your team is serious about agile software development practices, continuous integration (CI) should be a good friend of yours. Integrating code frequently helps you discover bugs and conflicts early in the development process so you can rather spend your time building the next kick-ass feature.
“Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.” Martin Fowler
In this blog post, I’ll explain why you should self-host your CI server on premise for your iOS/Mac development. Then I’ll give a comparison of Jenkins vs. Xcode Server.
Why should we self-host CI server?
“Why should we self-host CI server for our iOS/Mac projects when there are plenty of other SaaS services?” That’s a valid question your manager or colleague may ask you. While there are plenty of cloud-based CI services like Travis CI, Circle CI, Bamboo, BuddyBuild etc. you should consider the following:
Privacy
It may be a business requirement that your source code cannot leave the walls of your office. In this case, cloud-based CI is an absolute no-go!
Full control
Self-hosting CI gives the freedom (and obligations) to manage all hardware, software, possible plugins and extensions. This is not to say that cloud-based CIs don’t also provide you with that to a certain degree, but you won’t have full access to the physical machine.
Staying on top of the game
Every year, I get really excited (and bet you do too) for WWDC 2016 — rumor says it will be on the 6th of June. Apple regularly announces a new iOS/Mac version plus a handful of new APIs at the conference. So it’s likely you’ll spend your summer preparing your apps for these new OS versions, as well as building new features with the new cutting edge APIs. While Apple releases new beta versions of Xcode (bundled with iOS/Mac beta SDK) throughout the summer, you’ll need your CI server to use these beta versions too. Here’s where cloud-based CIs fall short. Honza Dvorsky in his great talk “Continuous Integration on the Edge” explained how “late” cloud-based CIs can be on offering the latest versions of Xcode. Theoretically speaking, based on the Pre-release Software document, anyone with an Apple ID can download and start using pre-release versions of Xcode (thumbs up for Travis CI who already offers beta version of Xcode 7.3 as of today). It may take hours/days/weeks for your cloud-based CI to provide you with the latest version of Xcode, which could be a blocker for your fast-paced team. Again, it’s not you who determines when you can build using the latest beta SDK, and are instead left waiting to find out when you can start.
If any of the above 3 facts sound like something your organisation does and needs, read on. Otherwise, just use cloud-based CIs for your iOS/Mac development, and use the money and space you saved from avoiding on-premise CI on a nice plant 🌻.
Cloud-based CIs are generally very easy to setup. Simply tell them where your code lives. They may run your jobs faster since they can split execution between multiple servers. BUT they may send you a bill with a hefty price tag, even though some folks, like Travis CI don’t charge anything for open-source projects. You’ll also lose the chance to connect your own physical devices and the full access to the actual machine that’s building your code. Moreover, you’ll be dependent on your CI server provider, which may or may not be to your advantage.
Jenkins vs. Xcode Server
In my previous blog post, I explained How we use Podio, Fastlane, and Jenkins for continuous iOS app delivery. However, there is a very appealing alternative to Jenkins called Xcode Server.
Honza Dvorsky also wrote an awesome series of tutorials on how to get started and become advanced with Xcode Server. I’ll explain why you should consider using Xcode Server as self-hosted CI for your next iOS/Mac project, and why it really caught my attention.
Cost & Setup
Jenkins is an open source, so you can get for free here. Xcode Server can be downloaded from the Mac App Store for $19.99. However if you’re registered in the Apple Developer Program, you can get a redeem code and download it free.
In the Continuous Integration and Code Coverage in Xcode session, Matt Moriarity suggests we use Xcode Server, because it’s easy to setup. And damn, he is right! Setting up Xcode Server takes only a few clicks, while setting up Jenkins is likely to give you a few hiccups in the process.
Interaction with CI
It probably won’t come as a surprise that Xcode Server is deeply and very well integrated with Xcode. Jenkins doesn’t have such a capability, and you have to interact with it via web UI or command line interface.
Code coverage
Code coverage is a metric, that indicates the percent of your code paths that have been exercised by test suites. This metric gives confidence that your codebase is less unlikely to contain bugs. Prior to Xcode 7 it was not possible to generate coverage for Swift code. In Xcode 7, Apple introduced a new coverage format called profdata (making gcov legacy), which now supports Swift code — yeah 🎉.
Showing the code coverage results in Xcode Server is as easy as opening the desired Integration and choosing the Coverage tab.
Jenkins doesn’t offer this luxury, leaving you with additional work. Jenkins offers the Cobertura plugin, which you have to point to xml file with your coverage data. The challenge is to convert data from profdata to Cobertura xml format. Luckily, there’s a Ruby gem called Slather, which has recently merged in a branch providing you with functionality to convert profdata to Cobertura xml format.
Triggers
It’s likely you’ll want to perform certain actions before and after your CI job runs. An example of such an action could be execution of Fastlane lane composed of a rich set of available actions.
Jenkins offers a wide range of trigger plugins. In Xcode 6, Apple has introduced triggers as an addition to Xcode Server. Triggers in Xcode Server allow you to execute an arbitrary shell script. This opens up the door to endless opportunities — like for example setting up a physical release button to start your integration 🚀 (see demo at 31:15).
For more hands-on experience with Xcode Server triggers, see this tutorial.
Result visualisation
Both CIs in question provide you with useful stats and charts for an overview of how your CI job did.
Jenkins dashboard can be configured to look something like this:
While Xcode Server dashboard looks something like this:
Apart from showing the results of your tests and code coverage, Xcode Server integration overview provides both error and warning count. You can also see static analysis issues.
Personally, I also really like that Xcode Server has a feature to share your stats on Big Screen, with an appealing and clear UI. This gives everyone in your team a real time snapshot for feedback on the health status of your code base while waiting for ☕️.
I love the idea of hooking up a lava lamp with a build server as a way to improve (and speed up) developers reactions to a broken build.
Luckily, both CIs provide you with REST API, enabling you to do so. See Jenkins REST API, Xcode Server API and Xcode Server SDK.
Conclusion
Putting Jenkins along side Xcode Server for a comparison, I see more advantages in using Xcode Server for iOS/Mac development. Apple invested heavily into this product in recent years and I hope we’ll see more cool stuff being released at WWDC 2016. Unless there’s a business requirement to use Jenkins, I have yet to find any reasons that make a case against using Xcode Server — so in closing, I urge you to give it a try.
