Managing Kubernetes with Kapitan

In my first post I explained a little bit the philosophy behind Kapitan and how it came to be.

Let kapitan take the helm of Kubernetes

In this post, I will give a more pragmatic introduction so that you can easily evaluate it and see if it fits your needs.

Kapitan is a tool to template files. It can be used to template things like text, documentation, scripts or yaml/json manifests. It was created to manage Kubernetes based deployments but it is flexible enough to be used in completely different contexts.

To get started with it, you can run it using docker or following these instructions.

For this post, I have also created a Katacoda scenario to allow you to play with it and take it for a run.

Introduction to Kapitan by ademaria | Katacoda

This post will focus on the use of Kapitan to manage Kubernetes deployments.

Assuming that you are all set and can run kapitan , let’s continue!

$ kapitan --version
0.22.3    # <-- Tested with this version: might be different!

Download the examples

Follow the instructions to download the examples and let’s use the example in the example-3 folder.

$ git clone https://github.com/ademariag/kapitan-examples --depth 1
$ cd kapitan-examples/example-3/

Let’s sail the Seven Seas!

This example will show you how to work with multiple environments. It comes with 2 environments: prod-sea and dev-sea .

You can run kapitan compile to verify that the compilation works

$ kapitan compile
Compiled prod-sea (0.43s)
Compiled dev-sea (0.45s)

Kapitan works by compiling the templates into their concrete form. By default, kapitan stores the compiled files in the compiled/ subfolder of the workspace.

compiled
|-- dev-sea
|   |-- README.md
|   |-- cod.md
|   |-- manifests
|   |   |-- cod-configmap.yml
|   |   |-- cod-deployment.yml
|   |   |-- namespace.yml
|   |   |-- sardine-configmap.yml
|   |   |-- sardine-deployment.yml
|   |   |-- tuna-configmap.yml
|   |   `-- tuna-deployment.yml
|   |-- sardine.md
|   |-- scripts
|   |   |-- apply.sh
|   |   |-- kubectl.sh
|   |   |-- setup_cluster.sh
|   |   `-- setup_context.sh
|   `-- tuna.md
`-- prod-sea
[cut]

Kapitan compilation is an idempotent operation. If none of the inputs have changed, you can expect no changes in the compiled files. In fact, let’s verify that the git status is clean

$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean

So far, nothing exciting. But you can see that there are several files in that compiled subfolder. Where do they come from? Let’s nuke the directory and run compile again.

$ rm -fr compiled/*     # <-- I shouldn't need to warn you..
$ kapitan compile
Compiled dev-sea (0.46s)
Compiled prod-sea (0.46s)
$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean

As you can see, the files are immediately recreated, and because kapitan compile is an idempotent operation, git status shows no changes.

So far, so good!

Targets and Inventory

Kapitan uses an inventory that represents the single source of truth of your deployments.

The inventory is based on a python library called reclass which is a fork of the amazing library by madduck.

The entrypoint for the inventory is a target. As you can see in the inventory/targets folder, at the moment there are 2 targets:
dev-sea.yml and prod-sea.yml

$ cat inventory/targets/dev-sea.yml
classes:
  - projects.minikube
  - components.tuna
  - components.cod
  - components.sardine
  - features.brine
  - features.canned
  - releases.v2.0.0
  - stages.development
parameters:
  target: dev-sea
  owner: developers

Aaaahh… YAML at last! Now we are talking!

I strongly suggest you to read more about reclass if you intend to discover its full potential.

Reclass essentially merges multiple yaml fragments by using a hierarchical Class inheritance: this way, reclass allows to reuse parameters across multiple targets.

Compare the content in prod-sea.yml and dev-sea.yml and you will see that the differences between them are minimal.

The “features.brine” class that you see in the target is nothing but a map to a actual yaml file: inventory/classes/features/brine.yml

$ cat inventory/classes/features/brine.yml
parameters:
  tuna:
    args:
      - --brine

Experiment 1: remove the “brine” feature

Let’s see what happens when you remove a class from a target. Remove the line corresponding to the class features.brine in the target dev-sea.yml and run kapitan compile again!

$ sed -i '/features.brine/d' inventory/targets/dev-sea.yml
$ kapitan compile
Compiled dev-sea (0.45s)
Compiled prod-sea (0.47s)
$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
modified:   compiled/dev-sea/manifests/tuna-deployment.yml
modified:   inventory/targets/dev-sea.yml
no changes added to commit (use "git add" and/or "git commit -a")

In this experiment, we have removed from the dev-sea target a class that represents the “brine” feature.

As you can see from the result of git status , this had the side-effect of also rewriting the compiled/dev-sea/manifests/tuna-deployment.yml

Diffing the file you can see that the operation results with the --brine command line argument being removed from the args list:

$ git diff compiled/dev-sea/manifests/tuna-deployment.yml
[cut]
@@ -14,7 +14,6 @@ spec:
         - args:
             - --verbose=True
             - --secret=?{ref:targets/dev-sea/tuna:ee958e1c}
-            - --brine
             - --canned
           image: alledm/tuna:v2.0.0
           name: tuna

Experiment 2: remove the “tuna” component

Let’s now try to remove the “tuna” component class. Similarly to the previous case, we just need to remove the “components.tuna” line from the dev-sea target.

$ sed -i '/components.tuna/d' inventory/targets/dev-sea.yml
$ kapitan compile
Compiled dev-sea (0.32s)
Compiled prod-sea (0.42s)
$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
        deleted:    compiled/dev-sea/manifests/tuna-configmap.yml
        deleted:    compiled/dev-sea/manifests/tuna-deployment.yml
        deleted:    compiled/dev-sea/tuna.md
        modified:   inventory/targets/dev-sea.yml
no changes added to commit (use "git add" and/or "git commit -a")

As you can see, removing the component causes the deletion of all the manifests and documentation associated to that component.

What can we learn from these experiments?

Already a lot:

• Changing the dev-sea target produced a change that only affected the dev-sea environment.
• Kapitan and git make it easy to assess which environment/target will be affected by a change.
• Both “cause” (the target change) and the “effect” (the deployment change) are visible to the reviewer.
• You can take a guess at what the change will do without having to actually deploy it to Kubernetes!

Let’s dissect the experiment

When we looked at the “brine” feature, we saw a YAML fragment.

$ cat inventory/classes/features/brine.yml
parameters:
  tuna:
    args:
      - --brine

Let’s see how it all ties together.

Inventory variable interpolation — quick intro

Reclass allows to refer yaml subtrees from other section of the inventory. i.e.

parameters:
  example:
    var: value
  other_var: ${example:var}

would result in an expanded inventory:

parameters:
  example:
    var: value
  other_var: value

kapitan inventory

First, we can learn to use the kapitan inventory command to see what is the actual inventory being computed by Kapitan.

You can look at the full inventory:

$ kapitan inventory
parameters:
  < lots of output >

You can drill down to a specific target:

$ kapitan inventory -t dev-sea
parameters:
< lots of output >

In our case, you can look at a specific sub-tree of the inventory:

$ kapitan inventory -t dev-sea -p parameters.tuna.args
- --verbose=True
- --secret=?{ref:targets/dev-sea/tuna|randomstr}
- --canned

What happens when we restore the “features.brine” feature?

$ git checkout -- inventory/targets/dev-sea.yml
$ kapitan inventory -t dev-sea -p parameters.tuna.args
- --verbose=True
- --secret=?{ref:targets/dev-sea/tuna|randomstr}
- --brine
- --canned

As you can see, the yaml fragment is injected into the parameters.tuna.args yaml list. But where is this list coming from?

kapitan searchvar

We can use the kapitan searchvar command to find where a specific value is defined:

$ kapitan searchvar parameters.tuna.args
./inventory/classes/features/canned.yml   ['--canned']
./inventory/classes/features/brine.yml    ['--brine']
./inventory/classes/components/tuna.yml   ['--verbose=${verbose}', '--secret=?{ref:targets/${target}/tuna|randomstr}']

As you can see, the parameters.tuna.args list is being referred to in these 3 files. You can already guess the canned.yml won’t be much different than the brine.yml that we have already opened.

Let’s look at the inventory/classes/components/tuna.yml instead:

$ cat inventory/classes/components/tuna.yml
parameters:
  releases:
    tuna: latest
  tuna:
    image: alledm/tuna:${tuna:release}
    release: ${releases:tuna}
    replicas: ${replicas}
    args:
    - --verbose=${verbose}
    - --secret=?{ref:targets/${target}/tuna|randomstr}
  kapitan:
    mains:
      - components/tuna/main.jsonnet
    docs:
      - components/tuna/docs/tuna.md

This inventory class defines the tuna inventory component. And this is also the first time we see a reference to jsonnet!

Pretty much everything in the inventory is up to you to define. You can organise it the way you want it. We have used “components/tuna.yml” but you can organise both the inventory files and the inventory content whichever way you want!

It’s no surprise that we find the “tuna” component subtree defined in this file. But I want to draw your attention to the “kapitan” section.

In fact, let’s use the kapitan inventory command to see the whole “parameters.kapitan” yaml sub-structure.

$ kapitan inventory -t dev-sea -p parameters.kapitan
 compile:
  - output_path: manifests
    output_type: yaml
     input_type: jsonnet
    input_paths: ${kapitan:mains}   # See notes*
  - output_path: .
    input_type: jinja2
    input_paths: ${kapitan:docs}    # See notes*
  - output_path: scripts
    input_type: jinja2
    input_paths: ${kapitan:scripts} # See notes*
docs:
  - components/docs/README.md
  - components/tuna/docs/tuna.md
  - components/cod/docs/cod.md
  - components/sardine/docs/sardine.md
mains:
  - components/namespace/main.jsonnet
  - components/tuna/main.jsonnet
  - components/cod/main.jsonnet
  - components/sardine/main.jsonnet
scripts:
  - components/scripts/apply.sh
  - components/scripts/kubectl.sh
  - components/scripts/setup_cluster.sh
  - components/scripts/setup_context.sh
vars:
  target: dev-sea

notes*: in reality running the command would show the interpolated version of these parameters which would be a bit confusing. I have replaced this values with the pre-interpolation reference.

Now we can finally understand what is happening: the components.tuna class in theparameters.kapitan sub-tree adds 2 files to be rendered.

kapitan:
    mains:
      - components/tuna/main.jsonnet
    docs:
      - components/tuna/docs/tuna.md

The components/tuna/main.jsonnet is a jsonnet type file to be compiled by the directive:

compile:
  - input_paths:  # this is a reference to ${kapitan:mains}
      - components/namespace/main.jsonnet
      - components/tuna/main.jsonnet
      - components/cod/main.jsonnet
      - components/sardine/main.jsonnet
    input_type: jsonnet
    output_path: manifests
    output_type: yaml
mains:
  - components/tuna/main.jsonnet
  - ...

and the components/tuna/docs/tuna.md is a jinja2 type file to be compiled by the directive:

compile:
  - input_paths:  # this is a reference to ${kapitan:docs}
      - components/docs/README.md
      - components/tuna/docs/tuna.md
      - components/cod/docs/cod.md
      - components/sardine/docs/sardine.md
    input_type: jinja2
    output_path: .
docs:
  - components/tuna/docs/tuna.md
  - ...

This concludes this post. In the new upcoming post, I will talk more about the actual deployment to kubernetes, plus we will go into details of how jsonnet and jinja templates work.

Please leave your feedback and let me know if you have comments. Clap to this post, star https://github.com/deepmind/kapitan and join us in the kubernetes slack channel #kapitan!

Continue to new post:

Kubernetes with jsonnet and Kapitan