Continuous Integration Journey of Tokopedia iOS Team
Join our journey implementing continuous integration into our development workflow since the very beginning up to today. We do it incrementally through different seasons.
Manual Era
Way back then before we even know what CI/CD is, just like living in the stone age, we’re doing all manually. We do compile directly to our testing device, re-download provisioning profile after registering a new device, took a while archiving and upload to iTunes Connect and so on. Sound familiar, eh? I believe most of us would do this, especially in our early stages of building an iOS app.
Back then, our team size is just a few, so taking those extra time doing those kinds of works doesn’t hurt anybody. Moreover, we are not forced to keep publishing updates; we control our releases.
All of those steps just okay, up until our team grows, codebase grows, and archive start took forever to finish, let alone slow or unstable connection during upload. Initially, we’re enjoying those extra free time waiting for our machine to complete the process until it took too long and we start bored waiting and waiting and waiting.
And then the team start to grow, more people joining the team, our product also growing at an incredible rate! Everything turned upside down in an instant. Product owners start pushing us to release more often with even more features packed in. All these manual steps start to be a burden more than ever. And then, we met Fastlane. It feels like a bit ray of daylight shuns upon us.
Hosted CI Comes to Rescue
Now with Fastlane in hand, we can run our iOS development phases smoother and more consistent! Moreover, we start incorporating CI on the cloud to aid us from wasting precious time. In the beginning, the only workflow automated is for upload release build to iTunes Connect, preparing for submission.
We are very excited! We can now do another (hopefully) more meaningful work than waiting for the archive to be done. And since that time, we keep improving our hosted CI integration and adding more and more lanes.
Up until now, we depend on our hosted CI so much. We build a debug mode app and upload them to HockeyApp on every push (now that’s what we call CI) and use the build result for some merging rules on GitHub. We do run DangerCI to enforce our believed code standards and wait for the build to be successful before the PR can be merged.
We are now more productive than ever! We can share the build result in HockeyApp to our remote co-workers. We have freed a ton of times building and archiving our app 🤘🤘
Our Hosted CI ‘Overwhelmed’
And the hiring frenzy starts! Our team grows, our throughput increased, more importantly, these great people are very productive! We bombard our hosted CI with jobs until our build has to be put on the queue. Our hosted CI started to billed us for extra build times used per month as we exceed the build time in our subscription! We did upgrade the plan, though. However, we have to tell the team this issue to help us minimize the cost by not pushing to often. The release pipeline is starting to fail on the stripping bitcode phase.
Besides that, a hosted CI build starts to be too long, almost 50 minutes. We tried many different approaches to solve this too, which I believe will be discussed in another post. Let’s focus on the CI itself; for now, it took forever! We become impatient waiting for a new build. Now we’re back to the age where we have to wait for the build. But, it runs on the cloud instead of our local machine.
With high hiring target in hand, double the current team size by the end of the year 😱 our hosted CI doesn’t sound like a wise choice any longer.
Transition Phase
While enduring the pain, we started our ‘side project’ of installing Jenkins. We read many articles written by some tech giants that says that they have a MacMini farm to run their automated build. We want to do the same, we gathered 3 of our still living MacMini and installed Jenkins there.
The main reason why we moved out to Jenkins is that Jenkins is scalable, we can add as much worker (I love the word ‘slave’ more) as we like. These slaves can be configured from the single control point, and we utilize resources it has as much as we can (see why I think ‘slave’ is more appropriate?). Above all, we only have to invest in the hardware cost, operational yes but summed up much less compared to hosted CI providers.
We experienced many setups and ended up with a MacMini as Jenkins server and added another iMac and MacPro as a slave. We also installed Grafana on every slave so we can monitor their health, and again the MacMini become the server for Grafana. Now, whenever builds are starting to wait for each other, we can easily add new slave. Now that our Jenkins are stable and can do all the work that was done in hosted CI, we are confident to kissing hosted CI our final goodbye.
Plus, we add much more capability into our Jenkins, we can now run a full UI Test on our app, which took nearly 5 hours to complete without hesitation every midnight. We also implemented several automated builds like testing release build, automated AppStore release workflow, etc. Above all, our debug build time reduced to about 20 minutes from almost 50 minutes 🤘!
Lesson Learned
Finally, I would say that implementing CI as early as possible is a wise move. You won’t feel much impact early on, but you’ll thank your past-self for doing the extra mile early. For those with a limited budget, there is a viable option, try Bitrise (not sponsored). They got a free plan that will enable you to free up your local machine from doing that build. More importantly, drive yourself for doing what it takes for a result. Get started early and adapt accordingly. Make it happen and make it better 🤘🤘!
