CI/CD worker nodes as virtual machines or K8s Containers?
When it comes to CI/CD deployments, what often doesn’t appear in documentation are the pros and cons of running your worker nodes as containers in a Kubernetes environment, or as (virtual) machines in a cloud environment.
You don’t need to adopt microservices solutions to potentially benefit from using Kubernetes environments to build apps. It’s also true that if you use CI/CD tooling that is K8s-aware like Argo, Tekton, and Jenkins X you won’t be able to maximize the use of their features.
Many of the tools used in CI/CD pipelines, such as Sonarqube, have prebuilt containers. Providing container images for tools is a great way to help people to try a tool without going through the investment of setup. But having a container is only half the story. Ideally, you need K8s config to ensure that the (secondary) artifacts such as code coverage data and outputs from lint are also retained.
If these considerations aren’t addressed, then will the CI/CD pipeline(s) enable acceleration of delivery and help eliminate first-time errors?
Our choice is a more advanced K8s configuration which ensures we don’t lose coverage data from our tools, but working/transient folders such as maven’s (.m2) repository aren’t persisted. Even with this is the fact we can’t treat containers as cattle while also reducing the risks of transient dependencies, etc. There’s also the potential of a great deal more elasticity in handling build demands. We could keep things simple and sacrifice the benefits containers offer, but this approach means the only gain is preinstalled software. Most CI/CD tools can have their deployment automated without too much effort, or manually deployed and imaged.
As an observation, note that GitHub Actions tends to steer people towards having VMs rather than containers for the workers. Sure, with some effort you could realize the workers as containers, but a K8s Operator for GitHub Actions would make for a powerful solution (an unofficial implementation can be found here).
Clearly the benefits of both approaches are pretty strong. So how do you decide? We can use a stress test also sometimes called a decision matrix — an idea I’ve written about before ( Decision Matrix aka ‘Stress Test’ as a vehicle to make decisions easier).
As you can see from our matrix there is a fair case, but some significant questions need to be addressed.
As part of the Oracle Reference Architecture (RA) team we’ve taken this further and outlined these ideas, then mapped them onto the different cloud CI/CD architectures to help make it easy to determine which RA will fit your needs best. This allows you leverage the created guidance and automation to bootstrap the process. Take a look at the fleshed-out view in the Architecture Center.
