Configuration Management on Kubernetes — Kapitan
In order to manage configuration on Kubernetes (not Kubernetes installation but configuration), one needs to associate 3 pieces of information: cluster, namespace and manifests.
Cluster and namespace determines where. They constitute a kind of address where changes will be deployed.
The manifests are what. They describe the changes in the desired state of the target cluster.
In addition we want to make it simple to insert the output from our CI/CD pipeline as a configuration change (ie. new docker image tag).
After building a custom tool chain for Weyv, I decided to try Kapitan again and see if I could make it all work. Our current tool chain is based on Helm, the values files are templated to incorporate changes coming from our CI/CD pipelines using Ansible then charts are rendered as manifests via helm templating. The configuration values are also separated by convention in cluster, namespace and component in a folder structure.
Let’s walk through how we would do that with Kapitan.
Inventory
The first concept we’re going to use is the inventory: “This is a hierarchical database of variables that are passed to the targets during compilation.”
Concretely, the inventory allows us to define values at different levels: cluster, environment, component, … and to combine them into targets, giving us reusability of values.
A component class in the inventory describes how to produce the results.
A version class is the trick we use to make it easy to plug our CI/CD pipeline into Kapitan: just output a simple file with the new information required to patch the target.
UPDATE: With the availability of refs of type plain, version target is not needed anymore. We can just store the image tag as plain refs in a new hierarchy:
Components
Components are the definitions of resources you want to deploy on a cluster.
Here is an example of the api component which is composed of a deployment and a service.
The deployment is defined with helper functions from the kube.libjsonnet library:
The service can also be defined directly in jsonnet:
Targets
A target is the combination of all the information required to produce an adequate change in the right place. It can include information related to a whole namespace or just one component as shown below.
Kapitan makes it easy to combine classes so I can define each component separately as well as define a macro-class that combines multiple components for targets. For example I could define targets to deploy api, mongodb and web components in the dev namespace of our k8s-dev cluster. Then I can define a class that invokes the 3 above classes and use that class to deploy the whole dev namespace with all it’s required components.
For example, the target k8s-dev-dev-api.yml which stands for cluster k8s-dev, dev namespace, api component , looks like the following:
Let’s say I also have a target k8s-dev-dev-web.yml for my web server, I can create a class k8s-dev-dev-backend that will inherit both api and web classes.
Conclusion
It seems Kapitan is a really good tool to solve our configuration management problems. Though since our current configuration relies on Helm charts, we will need a solution to use/transform/import them to minimize the migration costs.
