Bonny: Extending Kubernetes with Elixir — Part 2

Arrrr, step away from the helm, gopher.

If you aren’t familiar with Operators, read Part 1. All of the source code for this tutorial is located here.

Before getting started building an Operator with Bonny, I’m making a few assumptions:

  • You are familiar with the idea of Kubernetes Operators. See Part 1.
  • You have a docker/quay.io account.
  • You have a running Kubernetes cluster. Start here if you don’t.

Ok, let’s break production.

Hello Operator

We are going to create a “Hello Operator.” This operator will have a single custom resource definition named Greeting that will be composed of a Kubernetes Deployment and Service running this greeting-server

Any Mix project can be turned into an operator, but let’s create a new one:

mix new hello_operator

Edit your mix.exs to include these dependencies:

And now fetch dependencies:

mix deps.get

Now we are going to create our first controller. An operator can have many controllers, and a controller can create many Kubernetes resources.

Throughout this series, we are going to create a Greeting controller that will create deployments and services.

mix bonny.gen.controller Greeting greetings
* creating lib/hello_operator/controllers/v1/greeting.ex
* creating test/hello_operator/controllers/v1/greeting_test.exs

The first argument above is the Elixir module/CRD name. The second argument is the plural form.

Add the following to your config/config.exs file to load your controller:

For a full list of options, see the README.

Controllers are the only required configuration option. Bonny defaults to using the service account assigned to the pod. Here we are setting our operator up to run from outside the cluster for development purposes.

At this point, you have a functioning Kubernetes Operator. It doesn’t do anything interesting, but you’ve got one.

You’re the best.

Let’s compile our app and generate a manifest to see it in action.

# We need to compile our app so that the mix task can pick up our controller implementations
mix compile
# Generate a manifest. If you were deploying to production supply the --image flag
mix bonny.gen.manifest
# Deploy RBAC and CRDs
kubectl apply -f ./manifest.yaml
# Start our operator
iex -S mix

Open up another terminal and create some greetings:

kubectl apply -f https://raw.githubusercontent.com/coryodaniel/hello_operator/master/greetings.yaml
An example Greeting resource

You should see a flurry of debug and info log messages in your IEx session.

Now let’s take a peek at the Greeting controller in lib/hello_operator/controllers/v1/greeting.ex

@moduledoc removed for conciseness

This is the full API for a CRD controller. Bonny introspects much of the configuration, but everything is overridable using annotations.

Annotations:

@scope the scope of the CRD. Options are :namespaced (default) and :cluster. It determines scope at which an instance of the operator will manage lifecycles.

@names are the different name variations of the CRD. This should be a map with four atom keys:

  • kind: Must be capitalized. This name will be used in the kind field of a Kubernetes YAML file to create an instance of this CRD.
  • plural: The plural name will be prepended to the group to identify this CRD in the group. You can also use this name with kubectl kubectl get greetings.
  • singular: The name of the resource. Used as the singular form in kubectl kubectl get greeting/hello-server.
  • short_names: List of “short names” that kubectl will recognize for this resource. E.g.: kubectl get gre and kubectl get greet

@group is the Kubernetes API group the CRD belongs to. This is used by Kubernetes as a part of the REST URL for the CRD. This will default to your operator name + “example.com”. You can set it globally in config.exs or on a controller-by-controller basis here.

@version Set the Kubernetes API version for this resource. It defaults to the down-cased version of the second to last module name. E.g: HelloOperator...V1.Greeting would be v1.

@rule is an accumulated attribute (you can call it multiple times). Each time a tuple should be provided. The rule defines a Kubernetes RBAC rule for this controller, effectively the other Kubernetes resources this controller will be accessing. The tuple format is {apiVersion, list(resources), list(verbs)}.

If a controller creates a Deployment and a Service, the rules would be:

@rule {"apps", ["deployments"], ["*"]}
@rule {"", ["services"], ["*"]}

In the above example, deployments are under the API group apps and want all (*) permissions. Some resources, like Services, aren’t under an API group so we use an empty string "".

Lifecycle functions:

The lifecycle is the logic of what to do with a CRD resource. The API is very simple.

There are three events that Bonny watches for from the Kubernetes API: ADDED, MODIFIED, and DELETED. Each event is dispatched to the equivalent function on the controller module:

  • add/1 called when a new CRD resource is created.
  • modify/1 called when a CRD resource is updated.
  • delete/1 called when a CRD resource is deleted.

All three functions receive the same map payload (below) and return :ok | :error. Actual error handling/cleanup should happen in your function implementation. The return status is to support metrics collection.¹

Stay tuned for the next post when we’ll add functionality for deploying a greeting-server² application and an HTTP service to our operator.

Ready to make this operator do something interesting? Read Part 3!

Footnotes:

  1. Metrics collection isn’t currently implemented. Feel free to send a PR!
  2. Did I write an Elixir Kubernetes framework, then make an example app in Go?! Yes, I’m a complicated person.