This post is part of run Kubernetes components one by one series.
This post describes the steps to run kubelet standalone(no other Kubernetes components, just a binary)using Container Runtimes Docker Engine, this is the first post that focus on to make kubelet standalone startup and create pod successfully, the second post tries to explain the CNI part involved in kubelet.
kubelet is an agent that runs on each node in the cluster. It makes sure that containers are running in a Pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn’t manage containers which were not created by Kubernetes.
Anyway, all information here are based on my personal learning, so if there are any mistakes, feel free to point them out, I am very happy to see that!
Ok, let’s start the journey!
I wanted to build kubelet from source code and run it on my Mac, but unfortunately, kubelet doesn’t support run on Darwin for now, as you can see, it is aborted on Darwin.
foreverXT@love ~/git/github/kubernetes
$ ./hack/local-up-cluster.sh
+++ [0805 14:51:59] Building go targets for darwin/amd64
k8s.io/kubernetes/hack/make-rules/helpers/go2make (non-static)
+++ [0805 14:52:27] Building go targets for darwin/amd64
k8s.io/kubernetes/cmd/kubectl (non-static)
k8s.io/kubernetes/cmd/kube-apiserver (static)
k8s.io/kubernetes/cmd/kube-controller-manager (static)
k8s.io/kubernetes/cmd/cloud-controller-manager (non-static)
k8s.io/kubernetes/cmd/kubelet (non-static)
k8s.io/kubernetes/cmd/kube-proxy (static)
....
deployment.apps/coredns created
service/kube-dns created
coredns addon successfully deployed.
kubelet is not currently supported in darwin, kubelet aborted.
kubelet is not currently supported in darwin, kube-proxy aborted.
Create default storage class for
storageclass.storage.k8s.io/standard created
Local Kubernetes cluster is running. Press Ctrl-C to shut it down.
So we need a VM, thanks for the Cloud, we can easily get a free VM on the Cloud Providers, I am using OCI(Oracle Cloud Infrastructure) in this page, as OCI can provisioning always-free VM to use, although the VM spec is very small(just 1 CPU and 1G memory), but it is sufficient for the kubelet, so very cool!
Let’s start.
Basically, we need the following tools installed onto the VM
- Docker Engine
- cri-dockerd
- Network plugin
- kubelet
Docker Engine
We need to install Docker Engine by the following command. If you need run docker with other userid, you can just add that user to group docker
(using root)
$ yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
$ yum install docker-ce
$ systemctl enable docker
$ systemctl start docker
$ systemctl status docker
docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2022-08-05 07:19:20 GMT; 3s ago
....
As you may know, Kubernetes has removed dockershim as of release 1.24, the dockershim is a component that is used for direct integration with Docker Engine, so for using Docker Engine as the container runtime for kubelet 1.24, we need to install cri-dockerd, an adapter provides a shim for Docker Engine that lets you control Docker via Kubernetes Container Runtime Interface.
The steps to install cri-dockerd are mentioned here https://github.com/Mirantis/cri-dockerd
(using root)
$ wget https://storage.googleapis.com/golang/getgo/installer_linux
$ chmod +x installer_linux
$ /installer_linux
$ source ~/.bash_profile
$ yum install git -y
$ git clone https://github.com/Mirantis/cri-dockerd
$ cd cri-dockerd
$ mkdir bin
$ go build -o bin/cri-dockerd
$ mkdir -p /usr/local/bin
$ install -o root -g root -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd
$ cp -a packaging/systemd/* /etc/systemd/system
$ sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service
$ systemctl daemon-reload
$ systemctl enable cri-docker.service
$ systemctl enable --now cri-docker.socket
In my case, the last two commands failed with error
Failed to enable unit: Unit file cri-docker.service does not exist.
Failed to enable unit: Unit file cri-docker.socket does not exist.
Fixed by command restorecon(seems some label issues in the two files, not sure?)
$ restorecon /etc/systemd/system/cri-docker.service
$ systemctl enable cri-docker.service
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /etc/systemd/system/cri-docker.service.
$
$ restorecon /etc/systemd/system/cri-docker.socket
$ systemctl enable cri-docker.socket
Created symlink /etc/systemd/system/sockets.target.wants/cri-docker.socket → /etc/systemd/system/cri-docker.socket.
$ systemctl start cri-docker
OK, cri-dockerd is installed successfully, we need to install a Network plugin. A Container Runtime, in the network context, is a daemon on a node configured to provide CRI(Container Runtime Interface) Services for kubelet. In particular, the Container Runtime must be configured to load the CNI(Container Network Interface) plugins to implement the Kubernetes network model.
Install Network plugin can follow the steps mentioned here https://github.com/cri-o/cri-o/blob/main/contrib/cni/README.md
$ git clone https://github.com/containernetworking/plugins
$ cd plugins/
$ git checkout v1.1.1
$ ./build_linux.sh
$ mkdir -p /opt/cni/bin
$ cp bin/* /opt/cni/bin
Very simple, right!
Ok, Container Runtime preparation is done. Now, we will install kubelet, to simplify the installation, I will download the last version of kubelet binary directly from https://dl.k8s.io/v1.24.3/bin/linux/amd64/kubelet
$ wget https://dl.k8s.io/v1.24.3/bin/linux/amd64/kubelet
$ chmod +x ./kubelet
For running kubelet, we need to create a KubeletConfiguration object, for each properties, can check https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/ for details.
$ mkdir configs
$ cd configs
$ cat kubeletConfigFile.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
enableServer: false
staticPodPath: /home/opc/k8s/kubelet-static-pod
readOnlyPort: 10250
failSwapOn: false
podCIDR: 10.241.1.0/24
authentication:
anonymous:
enabled: true
webhook:
enabled: false
authorization:
mode: AlwaysAllow
Ok, it is time to run kubelet
./kubelet --config=/home/opc/k8s/configs/kubeletConfigFile.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/cri-dockerd.sock
You see, the container-runtime-endpoint option is pointing to the cri-dockerd we installed previously.
Now, kubelet is running and watching the directory specified by staticPodPath property, in my case, the directory is /home/opc/k8s/kubelet-static-pod, that means if we put the POD yaml into that directory, kubelet will create the POD for us automatically.
$ ps -aux | grep kubelet
root 717002 1.1 6.9 1714764 48908 pts/2 Sl+ 08:37 0:11 ./kubelet --config=/home/opc/k8s/configs/kubeletConfigFile.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/cri-dockerd.sock
root 724710 0.0 0.1 12136 1188 pts/0 S+ 08:54 0:00 grep --color=auto kubelet
[root@instance-20220803-1159 configs]#
Let’s try
$ pwd
/home/opc/k8s/kubelet-static-pod
$
$ ls
nginx.yaml
$
$ cat nginx.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /var/log/nginx
name: nginx-logs
- name: log-truncator
image: busybox
command:
- /bin/sh
args: [-c, 'while true; do cat /dev/null > /logdir/access.log; sleep 10; done']
volumeMounts:
- mountPath: /logdir
name: nginx-logs
volumes:
- name: nginx-logs
emptyDir: {}
After we create the ngnix.yaml, then you will see kubelet starts to create it
^@I0805 08:40:32.231689 717002 topology_manager.go:200] "Topology Admit Handler"
I0805 08:40:32.432222 717002 reconciler.go:270] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-logs\" (UniqueName: \"kubernetes.io/empty-dir/83760d52afdfe74389411f38b257d1c5-nginx-logs\") pod \"nginx-instance-20220803-1159\" (UID: \"83760d52afdfe74389411f38b257d1c5\") " pod="default/nginx-instance-20220803-1159"
^@
After some time, you will see the the containers defined in the nginx.yaml is running.
Super Amazing, isn’t it?
## kubelet GET pods api returns the pod created
$ curl http://localhost:10250/pods/ | jq . | head
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2640 0 2640 0 0 429k 0 --:--:-- --:--:-- --:--:-- 429k
{
"kind": "PodList",
"apiVersion": "v1",
"metadata": {},
"items": [
{
"metadata": {
"name": "nginx-instance-20220803-1159",
"namespace": "default",
"uid": "9c93cee38fd11dde61debf3b9705fc69",
## Pod containers were created
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6fcd9ccf10e4 busybox "/bin/sh -c 'while t…" 22 minutes ago Up 22 minutes k8s_log-truncator_nginx-instance-20220803-1159_default_83760d52afdfe74389411f38b257d1c5_0
bac90cef90f4 nginx "/docker-entrypoint.…" 22 minutes ago Up 22 minutes k8s_nginx_nginx-instance-20220803-1159_default_83760d52afdfe74389411f38b257d1c5_0
0e5c748e5628 k8s.gcr.io/pause:3.6 "/pause" 23 minutes ago Up 23 minutes k8s_POD_nginx-instance-20220803-1159_default_83760d52afdfe74389411f38b257d1c5_0
If we remove the nginx.yaml from /home/opc/k8s/kubelet-static-pod, then the pod will be deleted automatically.
## kubelet log
I0902 06:09:29.587821 2013106 reconciler.go:211] "operationExecutor.UnmountVolume started for volume \"nginx-logs\" (UniqueName: \"kubernetes.io/empty-dir/9c93cee38fd11dde61debf3b9705fc69-nginx-logs\") pod \"9c93cee38fd11dde61debf3b9705fc69\" (UID: \"9c93cee38fd11dde61debf3b9705fc69\") "
I0902 06:09:29.596492 2013106 operation_generator.go:890] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c93cee38fd11dde61debf3b9705fc69-nginx-logs" (OuterVolumeSpecName: "nginx-logs") pod "9c93cee38fd11dde61debf3b9705fc69" (UID: "9c93cee38fd11dde61debf3b9705fc69"). InnerVolumeSpecName "nginx-logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue ""
I0902 06:09:29.688903 2013106 reconciler.go:399] "Volume detached for volume \"nginx-logs\" (UniqueName: \"kubernetes.io/empty-dir/9c93cee38fd11dde61debf3b9705fc69-nginx-logs\") on node \"instance-20220803-1159\" DevicePath \"\""
## GET pods api call returns emtpy
$ curl http://localhost:10250/pods/ | jq . | head
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 64 100 64 0 0 12800 0 --:--:-- --:--:-- --:--:-- 12800
{
"kind": "PodList",
"apiVersion": "v1",
"metadata": {},
"items": null
}
## container were removed.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
[root@instance-20220803-1159 kubelet-static-pod]#
OK, we have a working kubelet standalone now, I will explain more on the CNI part in the next post.
Thanks for reading.
Check other posts of this series on Kubernetes 1.24+ components one by one series | by Elie | Medium