
Time for Jenkins to Retire?
Martin Fowler’s explains the idea of Continuous Integration best in his tongue-in-cheek CI Certification program. I summarize:
- Your team pushes code to a mainline.
- Automated build and test runs on said code.
- When the build fails, the build status returns to green (“passing”) within 10 minutes.
These are simple ideas: many teams want it, but some do not understand it. Of those who do, only a minority achieve it. There are a multitude of reasons.
In this article we will focus only one technical aspect: the CI Server. It is responsible for orchestrating build and test on each mainline push. In particular, we lay out the pros and cons of using Jenkins CI as the CI Server.
What is Jenkins?
Jenkins CI is an open source automation server that dominates the CI space. According to Datanyze, it owns just under 70% of the CI server market.
It was created in 2011. It is written in Java. It supports a dynamic plugins, and has a massive plugin ecosystem.
It has a master/agent architecture. For small projects, you just need to run a single master without any agents at all. Scaling up means attaching more machines as Jenkins agent nodes. Administration and user-space configuration is done through a mature Web UI.
The Good
There are strong reasons for Jenkins to rise to the top.
Ease of installation: Download a “.war” archive and just run it. In the age of docker-based deployments, it’s a docker run away.
Ease of configuration: Setting up a new Jenkins job is just a few clicks on the Web UI; no coding required. Everyone can use it, not just coders.
Plugin ecosystem: Almost all basic use cases and features are covered either in Jenkins core or by a free plugin.
Flexibility: It’s extremely customizable. E.g. you can:
- Run different test jobs on specific kinds of agent hardware.
- Send jobs to run a docker container on an agent
- Schedule jobs on Kubernetes
It’s not always easy or clean, but almost always possible.
Mindshare: Jenkins “experts” exist for hire to push Jenkins to its limit and beyond. Many software engineers are most comfortable with Jenkins.
The Bad
Jenkins has its weaknesses. Many are only apparent as a team and project grows in size, and as CI requirements increase in complexity.
Administrative load: Demand for CI capacity and complexity tends to grow super-linearly to size of your development teams [0]. You can hire “DevOps/Build” Engineers who know Jenkins inside-out, but this does not scale. These are often software engineers who could otherwise be producing alternate valuable work. As a Jenkins setup gets complex, it falls to a finite group of experts to implement new CI requirements. This is a hard cap on development velocity.
Poor stability: As Jenkins deployments grow just modestly, stability unreasonably suffers more. I attribute this to the hot-plugin system and to the aging codebase that has extended beyond its original vision. Because CI is so central to modern development workflows, downtimes are very costly (think software engineering wages).
Variable Plugin Quality: While there are plenty of plugins available for download, the quality is spotty. At best, flakey plugins fail to function as advertised. At worst, they destabilize the Jenkins system. You need to know what particular plugin / Jenkins version combo will cause problems from first hand experience.
Primitive Resource Management: Agent nodes are configured with a number of homogenous “slots”. No awareness of any actual server resources like CPU, memory or disk.
Groovy scripting language: Used for defining Jenkins pipelines which invoke other jobs programmatically. It’s not known to be very effective or productive and lacks tools / IDE integration. In my experience, much of Jenkins pipeline development is done directly through the Web UI.
Aging Web UI: NOT your modern React, Angular or even Ruby-on-Rails modern Web front-end. A flashy UI never seemed to be a priority. Users nowadays expect (and deserve?) more.
Poor configuration source control: A member of your team accidentally messes up a job configuration. You would not know who did it, when it happened, nor why the change was made. There is no practical way to peer review changes. There is no way to rollback to any previously good configuration. [1]
Conclusion
I do not think Jenkins CI is the obvious choice. However it can still be a reasonable choice. Because of its incumbent position, you can get quite a bit of mileage from it with its shallow on-ramp and broad feature set.
But remember, cost to benefit benefit ratio degrades as:
- Team size increases.
- Development velocity increases.
- Development processes become decentralized (as is best practice). This includes CI, CD (continuous deployment), QA, Release functions.
- Build and test resource needs becomes large and spikey (think release crunches, hotfixes).
If Jenkins is retiring, who will fill his shoes?
The CI product ecosystem is not a one-horse race. TeamCity, CircleCI and TravisCI are all strong contenders. All are SaaS offerings with free tiers for open source projects. All have configuration source control by design. When looking for a Jenkins alternative, I would recommend evaluating all three of these products.
The key criteria should be: Is the kind of CI workload (builds, tests, may be deploy, etc.) you wish to run today and for the foreseeable future both possible and easy? Can your rank-and-file developer self-serve as much as possible?
For those willing to consider the bleeding edge: another notable project is Argo, an open source container workflow engine for Kubernetes. It’s still early days, but it has the potential to become a standard for large-scale CI clusters.
Notes
[0] I have seen this anecdotally from personal experience. It is justified if you consider the following hugely simplified model (N = number of developers):
- Total number of features / components= O(N)
- Number of pairwise integration tests = O(N²)
- Code merge frequency = O(N)
- Naïve CI system load = Product of the above = O(N⁴)
In reality we will be able to cut down much of this through being smart about consolidating integration tests, using team/feature branches, etc. However I’d contend that few software engineering organizations manage to tame this to O(N) or less.
[1] There are plugins that will attempt to provide some of the above the features, but none are fully satisfactory, like a standard git-based approach might be. Recent incarnations of Jenkins do support loading pipeline definitions from within your source code repository. However you cannot get away from having potentially large amounts of functionality configured from within the Web UI alone.
Originally published at infinitydevops.com on September 3, 2018.