A quick introduction to Kubernetes Operator
The concept of Kubernetes Operator is not something new as it was introduced end 2016 by CoreOS in the Introducing Operators blog post.
As they defined it:
An Operator is an application-specific controller that extends the Kubernetes API to create, configure, and manage instances of complex stateful applications on behalf of a Kubernetes user
To make it simple, an Operator is a process (running in a Pod) that uses custom Kubernetes resources (resource that does not exists in Kubernetes by default) and communicates with the API Server to automate the workflow of complex applications.
Mid 2018, RedHat and the Kubernetes community released the Operator Framework to simplify the development of new Operators using the Go programming language.
SDK for building Kubernetes applications. Provides high level APIs, useful abstractions, and project scaffolding. …
A lot of Operators have been created, just check out this list https://github.com/operator-framework/awesome-operators
While Operators are great to manage stateful applications, they can also be used to perform special tasks for stateless applications as well. For instance, the Kanary Operator created by David Benque and Cedric Lamoriniere, allows to ease the Canary deployments.
Contribute to AmadeusITGroup/kanary development by creating an account on GitHub.
If you are not fluent in Go, Zalando released a framework to build Operators using Python. The purpose of this article is to introduce this new framework and to build a sample Operator with it.
Kubernetes Operator Pythonic Framework
Kopf (it’s the short name of this framework) is part of the Zalando-incubator github repository.
A Python framework to write Kubernetes operators in just few lines of code. - zalando-incubator/kopf
Yes, Zalando, like in https://www.zalando.com, those guys do some great stuff under the hood
This project is well documented as you can see in https://kopf.readthedocs.io, so let’s dive in and start building a sample Operator using Kopf !
Creation of a simple operator in Python
This Operator will define its own Database resource and trigger the creation of a Pod and a Service each time a resource of this type is created. The Pod will be based on the mongo or mysql Docker image depending upon the value of the type field specified in the Database resource. Not crystal clear yet ? Let’s have a closer look.
Create a CustomResourceDefinition (CRD)
An Operator usually defines its own Kubernetes resources on which it operates. The definition of a new Kubernetes resource is done with a CustomResourceDefinition object. Below is the definition of our new resource, of type Database. It is a simple object in which we only define the additional Type property.
$ cat <<EOF > crd.yml
- name: v1
- name: Type
description: The type of the database
EOF$ kubectl apply -f crd.yml
This new object now exist in the cluster, we need the Operator process to use it.
The Operator handler
In order to track the creation (and deletion) of a Database object, we use the following python script.
This one is quite simple, let’s detail the main parts:
- L5–6: define the handler triggered each time a Database object is created
- L10–14: make sure a type is defined in the Database object
- L16–20: define one template for the database Pod and one for the Service that will be used to expose the Pod
- L24–33: add additional fields in the Pod and Service template based on the type specified in the Database object
- L36–37: link both Pod and Service to the original Database object to that the deletion of the Database triggers the deletion of those guys as well
- L39–48: call the Kubernetes API Server to create the Pod and Service
- L54–55: define the handler triggered each time a Database object is deleted
Building an image for the Operator
The Operator will run in a Pod and thus needs to be packaged in a Docker image, we will use the following Dockerfile.
COPY handlers.py /handlers.py
RUN pip install kopf
CMD kopf run --standalone /handlers.py
The image can then be built and pushed to the Docker Hub with the usual commands:
$ docker image build -t lucj/op-db:latest .$ docker image push lucj/op-db:latest
As the Operator needs the right to create resources in the cluster (Pod and Service in this case), we will first create a ServiceAccount.
$ cat <<EOF > sa.yml
EOF$ kubectl apply -f sa.yml
and bind this ServiceAccount to the cluster-admin role.
$ cat <<EOF > binding.yml
- kind: ServiceAccount
EOF$ kubectl apply -f binding.yml
Note: the cluster-admin role provides the full privileges on the cluster, in a production example we would use a role with the exact lists of actions needed by the ServiceAccount instead of the cluster-admin
Deploying the Operator
Let’s now deploy the Operator using the following specification :
$ cat <<EOF > operator.yml
- image: lucj/db-op:latest
EOF$ kubectl apply -f operator.yml
We can then verify it’s running fine :
$ kubectl get deploy,pod
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/op 1/1 1 1 5m54sNAME READY STATUS RESTARTS AGE
pod/op-5f578856fd-cmkv2 1/1 Running 0 5m54s
Let’s consider a Database object with the following specification:
$ cat <<EOF > mongo.yml
EOF$ kubectl apply -f mongo.yml
Let’s verify the creation of a Pod and a Service have been triggered:
$ kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/mongo-db 1/1 Running 0 2m39s
pod/op-5f578856fd-cmkv2 1/1 Running 0 9m10sNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 14m
service/mongo-db NodePort 10.105.157.68 <none> 27017:31480/TCP 2m39s
As the type is set to mongo in the Database object’s specification, the Pod is based on the mongo:4.0 image. We can use the NodePort associated to the Service to access the mongo Pod from our MongoDB Client, Compass here.
As the Pod and Service are defined as children of the Database object, they are deleted with it.
$ kubectl delete -f mongo.yml
database.zalando.org "mongo-db" deleted$ kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/op-5f578856fd-cmkv2 1/1 Running 0 11mNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 16m
We could also test with the creation of a Database object of type mysql, this would create a Pod based on the mysql:8 image and a NodePort Service exposing it.
This example used in this article is very basic but I hope it provides some useful information to get started with Kopf.
The code is available in a Gitlab repository should you want to give this guy a closer look https://gitlab.com/lucj/example-kopf-operator.
Are you considering to develop your own Kubernetes Operator ? I’d love to know which SDK you will go with ?