Bonny: Extending Kubernetes with Elixir — Part 1

Arrrr, I be takin’ over yer ship, gopher.

This morning I released Bonny, an Elixir framework for building Operators to extend Kubernetes.

The goal of Bonny is to provide a simple, expressive interface for extending Kubernetes, as well as:

  • reducing controller boilerplate so the focus is on resource lifecycles
  • handling Kubernetes YAML manifest creation including service account, RBAC, and deployment generation
  • supporting multiple controllers per operator
  • providing sensible safe defaults

Bonny isn’t the only operator framework¹, but it is the only one named after a pirate and written in Elixir.

In this multi-part post, we are going to create a Hello Operator that will be able to deploy multiple versions of a Greeting application.

What is an operator?

Before we build an operator, let’s go over some foundational terminology:

Custom Resource Definition (CRD):

A CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource.

You can think of this like a class in object-oriented programming or a table in a database.

Once created you will be able to access your custom resource definitions with kubectl like any other Kubernetes resource.

For example, given the Greeting CRD we will generate in this tutorial, then:

> kubectl get greetings
NAME AGE
hello-server 15h
hola-server 15h

Custom Resource:

A custom resource is an instance of a given CRD.

You can think of this as an object instance in object-oriented programming or a record in a database table. If a Greeting is your CRD, hello-server is your custom resource.

Custom resources are also exposed via kubectl:

> kubectl describe greetings/hello-server
Name: hello-server
Namespace: default
Labels: ...
Annotations: ...
API Version: hello-operator.bonny.test/v1
Kind: Greeting
Metadata:
Creation Timestamp: 2018-12-29T02:36:45Z
Generation: 1
Resource Version: 1217937
Self Link: /apis/hello-operator.bonny.test/v1/namespaces/default/greetings/hello-server
UID: 95abdb28-0b12-11e9-bd27-025000000001
Spec:
Greeting: Hello

Controller:

A custom controller is a controller that users can deploy and update on a running cluster, independently of the cluster’s own lifecycle. Custom controllers can work with any kind of resource, but they are especially effective when combined with custom resources. The Operator pattern is one example of such a combination. It allows developers to encode domain knowledge for specific applications into an extension of the Kubernetes API.

Ugh, documentation definitions. What alien species are these written for?

A controller controls the lifecycle of adding, modifying, and deleting the Greeting resources.

Operator:

An operator is a set of application specific controllers deployed on Kubernetes and managed via kubectl and the Kubernetes API. Operators enable the Kubernetes API to manage third-party software as native Kubernetes objects.

Kubernetes provides a number of APIs to extend the functionality and behaviour² of a cluster including webhooks, the Aggregation API, and Custom Resource Definitions.

The Operator pattern uses a combination of CustomResourceDefinitions, the Watch API, and one or more controllers managing the lifecycles of an operator’s custom resources.

Imagine being able to deploy a highly available ElasticSearch cluster or Prometheus with a few lines of YAML configuration.³

Want a way of allowing engineers to create SNS Topics or S3 Buckets using the same tool they deploy applications to Kubernetes with? AWS has an operator to manage AWS Resources.

Operators give you the power to create packages that allow other engineers to deploy sophisticated and/or complex systems in a declarative way.

If you are ready to build your first operator, check out Part 2!

Footnotes:

  1. Other operator frameworks include CoreOS Operators SDK, KubeBuilder, and Kooper
  2. I spelled it this way on accident. I blame Elixir.
  3. The ElasticSearch and Prometheus operators are not built using Bonny.