Should you build your own CI server?
A tale of two efficiencies
Introducing CI/CD into your software process is an absolute no brainer. The question, endlessly, is “What CI tool should I use?”. There are so many to choose from that the effort in searching alone can be daunting. It feels like, once you have your server running, you’re married to it. The sheer amount of work it took to get the thing working reliably completely shuts down any desire to experiment with new approaches.
From here, we arrive into the world of SaaS offerings. Tools that lift most of the burden from your shoulders and charge only a license fee. They’re typically quick to get moving with, easy to configure and will have you up and running in no time.
For simple organisations making extensive use of cloud infrastructure, they’re a dream come true. For others, they’re not. In the following article, I’ll be providing some of the most common points when deciding to run your own CI or make use of a SaaS vendor solution.
Maintenance
Maintaining servers can feel like waste. Afterall, your organisation wants you to write new features and not spend time tweaking your existing infrastructure. It is a tale as old as time.
Engineer: “Look if we don’t do this, it will break and our service will go down”
Product Owner: “So build it so it doesn’t go down then!”
The truth is, neither is at fault. Some things need regular care and attention, but the Product Owner could be forgiven for not understanding this. The reasons are often buried beneath several layers of computing knowledge and explaining them takes more time than either party wants to spend.
If you do have a mature ops function in your org…
You might have engineers who know exactly how to keep things running, or you might have a dedicated operations department that are happy to keep the software ticking over. Either way, if you’ve got the people and the skills necessary, building your own CI server, with all the additional flexibility that will bring, might not be a bad idea. This is typically larger organisations that want to keep things in house.
But if you don’t…
Without the requisite skills, keeping things running can be hard. Really hard. Google refers to these things as “toil” — tasks that could be automated but still require human intervention. And without a good knowledge of how to keep things running, automating is going to be tricky. The more features you write, the more code you ship, the more complex your world will become. CI/CD should not be a burden, it should be an enabler, and you might want to consider using a SaaS solution to get you going.
Remember, many SaaS solutions offer a very low barrier to entry so you’re not tightly coupling your software estate to them. But bringing up your own CI/CD, hiring the individuals with the correct skills and forming an operations function is a big commitment. Once you’ve done it, you’re stuck with it. A SaaS solution can be temporary or permanent, the control is in your hands. For a small business, this is gold.
Funky Networking
Smaller companies find themselves, at an ever increasing rate, using cloud-based infrastructure. No need to build data centers or hire specialist Linux admins. Just let Amazon, Google or Microsoft do that for you. They’ve been doing it for years and they’re pretty good at it.
It isn’t all roses though…
Some organisations, especially large, old organisations, have got a much more complex set up. Servers that were created 35 years ago, mainframes that no one wants to touch or some VMs housed in someone's basement. These things crop up in larger companies because it is much more difficult to keep drift from being introduced.
How does this relate to CI/CD?
If you are thinking of using a SaaS solution, first look at what your solution will need to talk to. Do you have servers sat inside a private subnet in a datacenter somewhere halfway across the world? How are you going to enable this?
This is where you get into the world of bastion servers, VPNs and Jump boxes. It is complexity and it dramatically increases the barrier to entry for these products. For sufficiently large organisations, you may want to consider more than one CI/CD solution. Your newer products that hang out in the cloud might want to use a SaaS offering like CodePipeline or Buddy, while your older solutions can use something hosted in house. Again, complexity.
Cost — Time, Cash and Effort
The cost of spinning up and running your own CI server can be a little insidious. If you go with a tool like Jenkins, you’re in the world of open source. Jenkins is free, right? Well, kind of.
Jenkins itself is free, since you can use the software without having to pay anyone, but the engineering effort required to build, deploy and maintain Jenkins is most certainly not free. Unless you’re a charity with some very generous developers, you’re likely going to incur quite a hefty cost up front and an ongoing maintenance cost in the form of engineering time.
There is also the opportunity cost. If Jenkins is your first port of call, all the time you spend spinning up a server and tuning it to work correctly is time you’re not in the market. This is referred to as Cost of Delay.
So how does SaaS differ?
A SaaS solution comes with a subscription cost. These are often reasonable and justifiable, but they are ongoing costs. You’re getting up and running at break-neck speed, but you’re not investing in your own capabilities. Should that SaaS provider cease to exist, you’ve got to consider what it might do to your business.
The point here is that neither approach is free, and they both have their cost/benefit trade-off. With in-house CI/CD, you’re paying for ownership of your process but you’re paying with a lot of time, effort and money. If you haven’t got the right skills in-house, you’re also paying with new staff and recruitment costs. With SaaS, you’re paying a license cost at regular intervals and you’re not developing your internal capabilities, but you are getting to market quickly and that can often be the clincher for many organisations.
Security Vulnerabilities
It all comes back to skills and maturity. If you have a team or individuals within teams who are constantly patching security issues, this won’t be too scary for you. It is, however, an overhead worth considering when deciding your CI/CD approach.
I’ve used the Jenkins screenshot because I think this is the most mature solution to flagging and removing vulnerabilities that I’ve seen in a modern CI/CD solution. When you click on the “Manage Jenkins” button, you’ll be faced with the offending plugins. It’s tough to miss.
With a few button clicks, you’ll be upgraded to patched versions and you’re free from the known CVEs that exist in your CI system. This is an idyllic situation though. As we know, Jenkins plugins can clash with one another and minor version bumps can have unintended side effects. So while the removal of the vulnerability is one click, dealing with the fallout might be a night of reading server logs and working out why builds won’t run any more.
In short…
It comes down to organisational maturity and the problems you want to solve. Even very mature organisations might not be worried about their own internal CI/CD approach. Tools like Buddy and CodePipeline can do the job for you and the business can get on with the hard stuff. This is certainly the direction of travel in many other areas, with Serverless and SaaS becoming the “de facto” approach for organisations large and small.
