Building a Kubernetes Operator In Python With Zalando’s Kopf

Luc Juggery
Jun 25 · 6 min read

A quick introduction to Kubernetes Operator

The concept of Kubernetes Operator is not something new as it was introduced end 2016 by CoreOS in the Introducing Operators blog post.

As they defined it:

An Operator is an application-specific controller that extends the Kubernetes API to create, configure, and manage instances of complex stateful applications on behalf of a Kubernetes user

To make it simple, an Operator is a process (running in a Pod) that uses custom Kubernetes resources (resource that does not exists in Kubernetes by default) and communicates with the API Server to automate the workflow of complex applications.

Mid 2018, RedHat and the Kubernetes community released the Operator Framework to simplify the development of new Operators using the Go programming language.

A lot of Operators have been created, just check out this list https://github.com/operator-framework/awesome-operators

While Operators are great to manage stateful applications, they can also be used to perform special tasks for stateless applications as well. For instance, the Kanary Operator created by David Benque and Cedric Lamoriniere, allows to ease the Canary deployments.

If you are not fluent in Go, Zalando released a framework to build Operators using Python. The purpose of this article is to introduce this new framework and to build a sample Operator with it.

Kubernetes Operator Pythonic Framework

Kopf (it’s the short name of this framework) is part of the Zalando-incubator github repository.

Yes, Zalando, like in https://www.zalando.com, those guys do some great stuff under the hood

This project is well documented as you can see in https://kopf.readthedocs.io, so let’s dive in and start building a sample Operator using Kopf !

Creation of a simple operator in Python

This Operator will define its own Database resource and trigger the creation of a Pod and a Service each time a resource of this type is created. The Pod will be based on the mongo or mysql Docker image depending upon the value of the type field specified in the Database resource. Not crystal clear yet ? Let’s have a closer look.

Create a CustomResourceDefinition (CRD)

An Operator usually defines its own Kubernetes resources on which it operates. The definition of a new Kubernetes resource is done with a CustomResourceDefinition object. Below is the definition of our new resource, of type Database. It is a simple object in which we only define the additional Type property.

$ cat <<EOF > crd.yml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: databases.zalando.org
spec:
scope: Namespaced
group: zalando.org
versions:
- name: v1
served: true
storage: true
names:
kind: Database
plural: databases
singular: database
shortNames:
- db
- dbs
additionalPrinterColumns:
- name: Type
type: string
priority: 0
JSONPath: .spec.type
description: The type of the database
EOF
$ kubectl apply -f crd.yml

This new object now exist in the cluster, we need the Operator process to use it.

The Operator handler

In order to track the creation (and deletion) of a Database object, we use the following python script.

This one is quite simple, let’s detail the main parts:

  • L5–6: define the handler triggered each time a Database object is created
  • L10–14: make sure a type is defined in the Database object
  • L16–20: define one template for the database Pod and one for the Service that will be used to expose the Pod
  • L24–33: add additional fields in the Pod and Service template based on the type specified in the Database object
  • L36–37: link both Pod and Service to the original Database object to that the deletion of the Database triggers the deletion of those guys as well
  • L39–48: call the Kubernetes API Server to create the Pod and Service
  • L54–55: define the handler triggered each time a Database object is deleted

Building an image for the Operator

The Operator will run in a Pod and thus needs to be packaged in a Docker image, we will use the following Dockerfile.

FROM python:3.7
COPY handlers.py /handlers.py
RUN pip install kopf
CMD kopf run --standalone /handlers.py

The image can then be built and pushed to the Docker Hub with the usual commands:

$ docker image build -t lucj/op-db:latest .$ docker image push lucj/op-db:latest

RBAC Rules

As the Operator needs the right to create resources in the cluster (Pod and Service in this case), we will first create a ServiceAccount.

$ cat <<EOF > sa.yml
apiVersion: v1
kind: ServiceAccount
metadata:
name: db-operator
EOF
$ kubectl apply -f sa.yml

and bind this ServiceAccount to the cluster-admin role.

$ cat <<EOF > binding.yml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: db-operator
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: db-operator
namespace: default
EOF
$ kubectl apply -f binding.yml

Note: the cluster-admin role provides the full privileges on the cluster, in a production example we would use a role with the exact lists of actions needed by the ServiceAccount instead of the cluster-admin

Deploying the Operator

Let’s now deploy the Operator using the following specification :

$ cat <<EOF > operator.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: op
spec:
selector:
matchLabels:
app: op
template:
metadata:
labels:
app: op
spec:
serviceAccountName: db-operator
containers:
- image: lucj/db-op:latest
name: op
EOF
$ kubectl apply -f operator.yml

We can then verify it’s running fine :

$ kubectl get deploy,pod
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/op 1/1 1 1 5m54s
NAME READY STATUS RESTARTS AGE
pod/op-5f578856fd-cmkv2 1/1 Running 0 5m54s

Testing

Let’s consider a Database object with the following specification:

$ cat <<EOF > mongo.yml
apiVersion: zalando.org/v1
kind: Database
metadata:
name: mongo-db
spec:
type: mongo
EOF
$ kubectl apply -f mongo.yml
database.zalando.org/mongo-db created

Let’s verify the creation of a Pod and a Service have been triggered:

$ kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/mongo-db 1/1 Running 0 2m39s
pod/op-5f578856fd-cmkv2 1/1 Running 0 9m10s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 14m
service/mongo-db NodePort 10.105.157.68 <none> 27017:31480/TCP 2m39s

As the type is set to mongo in the Database object’s specification, the Pod is based on the mongo:4.0 image. We can use the NodePort associated to the Service to access the mongo Pod from our MongoDB Client, Compass here.

As the Pod and Service are defined as children of the Database object, they are deleted with it.

$ kubectl delete -f mongo.yml
database.zalando.org "mongo-db" deleted
$ kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/op-5f578856fd-cmkv2 1/1 Running 0 11m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 16m

We could also test with the creation of a Database object of type mysql, this would create a Pod based on the mysql:8 image and a NodePort Service exposing it.

Summary

This example used in this article is very basic but I hope it provides some useful information to get started with Kopf.

The code is available in a Gitlab repository should you want to give this guy a closer look https://gitlab.com/lucj/example-kopf-operator.

Are you considering to develop your own Kubernetes Operator ? I’d love to know which SDK you will go with ?

The Startup

Medium's largest active publication, followed by +504K people. Follow to join our community.

Luc Juggery

Written by

#DockerCaptain #Startups #Software #中文学生 Learning&Sharing

The Startup

Medium's largest active publication, followed by +504K people. Follow to join our community.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade