Initialize your AKS nodes with DaemonSets
It is a common pattern to run supporting software such as Malware scanners, Policy Enforcers etc. on (Azure Kubernetes Service) AKS worker nodes. This post is heavily inspired by the Kured project and walks you through the process of bootstrapping your AKS cluster with daemon-sets that make this possible.
DaemonSets in Kubernetes allow you to run a pod on every node; this is a good fit if you want to bootstrap new nodes and install software. You can configure the privileges that a DaemonSet runs with and tune the level of access you need your DaemonSet to have based on the tasks it needs to perform.
If you need to run commands on the host running your containers, you can use the nsenter command. As a prerequisite to being able to do this, you will need to make sure that your DaemonSet container has elevated privileges — you can do so by setting hostPID = true and privileged=true in your Daemonset YAML as shown below.
You also need to mount a host path to the DaemonSet so that you can copy the software installation script to the host. Additionally, mount a config map to the container with the installation script. This script can be modified to meet your needs. In this example, I am installing the cowsay program on all the nodes.
Next, lets have a look at how this actually works. I’ve created a generic image that runs the following shell script
The script first copies the installation script that has been mounted via the config map to the host (worker node). It also copies a wait script to the host. The wait script is needed when a new node starts up as the package manager usually locks any installation until it updates the system. The wait script holds the execution of the script until the lock is released. The scripts need to be run in the context of the host namespace and I am using nsenter to achieve that.
Finally, I have a simple Dockerfile that copies the scripts to a Debian container
We can now run the DaemonSet and ssh into the AKS nodes to see if cowsay was installed.
To summarise, we have run through the following steps to bootstrap your AKS cluster:
- Created a DaemonSet YAML and assigned elevated privileges to the container to enable it to execute commands as the host
- Created a ConfigMap with the script that needs to run on each node in the AKS cluster
- Validated that all the nodes in the cluster had the script installed and executed.
