Utilizing parallelism in Circle CI
Circle CI is a service that automates test builds. It hooks into github and whenever a commit is pushed, a “build” is started and runs the test suite however you like. This is extremely useful for making sure the changes you push don’t break anything.
The consequence of this is that there is now an extra requirement to getting your code merged. Usually this isn’t an issue because most companies have some code review process in place which generally takes longer than the test builds anyway. But what happens when your build start taking longer as you add more and more tests and as your app grows? That’s what happened to my team at work. Our test builds started trending upwards of 10min per build. This slowed down shipping overall.
I looked into fixing this problem by utilizing Circle CI’s parallelism functionality. The basic idea is to break up the test suite into N number of builds running in parallel. Now, this doesn’t necessarily translate into linear improvements. i.e 4x parallelism does not mean cutting the build time by 4. But it does offer noticeable improvements, where the benefits are more obvious as the test suite grows larger and large.
The end result was improving our builds by roughly 30% cutting 10–11min~ builds to 6–7min. My very rough estimate is we do 50 builds every single day. 50 builds * 3 minutes = 150 minutes a day. This saves us 2.5 hours of waiting for a build to finish per day, which all adds up in an impactful way. Not to mention relieving the mental annoyance of looking at your build and see a yellow dot. The caveat here is that we now require 3x containers for the build. Containers are resources shared by the company, so it’s worth measuring if using more containers is a valid trade off for improving your test builds.
See docs for setting up parallelism here https://circleci.com/docs/1.0/parallelism/.
