Today it’s hard to imagine building software without CI. The idea of running production builds on the same device you’re writing code on seems just… odd. And yet not so long ago everyone was doing it; that was the only way. The concepts of continuous integration and continuous delivery were first developed in the late 1990s; but CI/CD didn’t become mainstream until early 2010s.
Since then, the tooling evolved significantly. Long gone is the need to set up and maintain servers to have continuous integration. Instead, it is now integrated seamlessly with code repositories such as GitHub and GitLab. This article attempts to trace the evolution of CI/CD platforms from the early days till present.
Bias acknowledgement: I am building Digger — an open-source tool that allows you to run Terraform in your CI
The need for CI
We are used to it now; but 20 years ago, to have a dedicated server for builds would seem pretty counterintuitive. Why bother with such a complex setup? Why wouldn’t every engineer simply make the latest build on their machine and release by hand?
There are two problems with that. First is frequency: it turns out, again counterintuitively, that the more frequently you release your software, the more stable it gets — not the other way around. This is because the number of bugs grows exponentially with the number of changes in a release. So if you test changes in smaller batches, you have less bugs and catch them early; and catching bugs early also makes them easier to fix.
The second problem is consistency. Every engineer’s local environment is ever-so-slightly different. There is no way to guarantee that builds will run exactly the same everywhere; so builds are constantly breaking. A common solution to this was to have only one “release engineer” who runs all builds. But with this setup, you cannot have frequency. The very first CI servers probably were just scripts written by frustrated release engineers.
1st generation: CI server software
The first widely used CI software was CruiseControl, created in 2001 by Matt Foemmel at ThoughtWorks. You’d deploy CruiseControl to a physical server, just like pretty much all other server software at the time. There was no git back then, just SVN. Git was only created in 2005; AWS in 2006; GitHub in 2008.
In 2005 Kohsuke Kawaguchi at Sun Microsystems created Hudson — an alternative to CruiseControl that quickly overtook it in popularity because of superior UI and flexible job scheduling. In 2011 Hudson was renamed to Jenkins after a legal dispute with Oracle. Jenkins is still widely used today, especially on-premises without internet connectivity.
Another notable CI server is TeamCity from JetBrains, first released in 2006.
2nd generation: CI-as-a-service
In the early 2010s, cloud was the new hot thing. AWS, GCP and Azure founded few years earlier were growing like crazy. Software-as-a-service model made a ton of sense — because running the same server software on physical servers in everyone’s office didn’t.
The first piece of the stack to move to the cloud was code hosting, with GitHub and Bitbucket leading the way. Today we take this for granted, but it was nothing short of a revolution at the time: people from different countries could start a software project in a few clicks.
Naturally, CI/CD tooling followed. In 2011 TravisCI was founded and quickly became popular. It integreated nicely with GitHub and could be easily configured using a travis.yml file at the root of the repository.
CircleCI was also founded in 2011. Like Travis it offered GitHub integration and easy configuration via config.yml file. Circle invested heavily into performance and integrations; and they were the first CI to support Docker containers. Builds on Circle ran faster and more reliably than on Travis; as a result, Circle overtook Travis and is used to this day by many teams.
CI-as-a-service from cloud providers
As SaaS became the de-facto standard way of consuming and delivering software, major cloud providers couldn’t stand aside. Especially given that tools like CircleCI and Travis were themselves using AWS under the hood. So Amazon introduced CodeBuild in 2016; Google and Microsoft followed with Cloud Build and Azure DevOps respectively. These services are primarily used at enterprises who are already heavily invested in the AWS (or equivalent) ecosystem, with CI/CD being just one building block of many for the enterprise stack.
Specialised CI/CD platforms
Of all CI use cases, mobile apps and infrastructure are the trickiest. Mobile apps often need to target multiple platforms and OS versions for every build. And infrastructure-as-code tools like Terraform rely on state, so deployments are not independent from one another. So for some teams it makes sense to use different CI providers for different workflows.
For infrastructure-as-code, there are Terraform Cloud and Pulumi Cloud by Hashicorp and Pulumi respectively, as well as alternative products like Spacelift, Scalr or Env0. For mobile, some of the popular specialised CI platforms are Bitrise and CodeMagic.
3rd generation: integrated with git hosting
In 2015 Gitlab made a brilliant move — they’ve integrated CI/CD into their git hosting product. A year later in they outlined the vision for a seamless developer workflow in their master plan. Integrated CI/CD was likely the key reason for Gitlab’s rise in popularity and success of the cloud-based offering.
GitHub responded in 2018 by launching Actions, also seamlessly integrated with git hosting. With Actions they’ve also created an ecosystem of reusable steps, somewhat similar to layers in Docker. This made a huge difference — most of the routine scripting could now be reused. So Actions exploded in popularity, growing even faster than Gitlab CI did.
As of 2023 the standalone CI-as-a-service products are effectively obsolete. (Sorry Cirlce! you were great). The appeal of the integrated offering is just too strong; the ecosystem of Actions andis thriving. If your code is on GitHub, why use anything else?
Bias acknowledgement: I am building Digger — an open-source tool that allows you to run Terraform in your CI
