KoçSistem
Published in

KoçSistem

Kubernetes is Your Platform: Design Patterns For Extensible Controllers-My KubeCon Notes

Hi everyone,

I want to talk about my KubeCon Europe 2022 notes, Kubernetes is Your Platform: Design Patterns For Extensible Controllers session.

We know that, developing Kubernetes extensions is quickly becoming a mainstream practice to solve problems in a cloud native way.

But, when the business problem is complex, using a single controller quickly becomes a limit that developers are required to cross.

Let we learn that reusable design patterns derived from the concrete experience and the hard lessons learned on the field.

Let’s clarify some terms…

Controller: A single reconcile loop (such as the ReplicaSet controller)

Manager/Operator: A component that runs one or more controllers, like the kube-controller-manager

Custom Resource: an instance of a Kubernetes type defined by using Custom Resource Definitions. A custom resource is formed by metadata, spec and status.

Owner: Controller responsible for driving a custom resource from the observed state to the desired state. (Such as ReplicaSetController owns the ReplicaSet resources.)

Watch: Action of a controller that looks for resource under your control and other resources as well. (Such as ReplicaSetController watch for Pod resources.)

Orchestrate Many Controllers

This set of design patterns allows developer to break down composable, smaller controllers.

Controllers Watching The Same Object

A controller owns the CR, the second controller dynamically attach additional behaviours, eventually controllers can synchorize using finalizers or annotations.

USE WHEN…

  • You want to make progress after a precondition is met, such as machine is being deleted.

PROS…

  • Responsibility is well defined. Each controller does one task.
  • It is a battle tested pattern, such as garbage collector

CONS…

  • Limited orchestration options, such as finalizers
  • Behavioral dependencies between controllers are not well documented, they don’t surface in API.

Controller Watching Related Objects

A controller owns a parent CR and creates/deletes child CRs, another controller owns the child CR.

USE WHEN…

  • There are parent-child semantic relations between the objects you control.

PROS…

  • Proved effectiveness, it is in Kubernetes core.
  • Support sophisticated variants, such as history of changes, rollbacks

CONS…

  • Not trivial to debug, why is the resource not progressing…
  • Not trivial to evolve, such as change in a resource might impact more that one controller.

API Contracts

It is a variant of the controller watching related objects pattern where the second CR is pluggable and it is expected to have a set of well known fields (the API contract).

USE WHEN…

  • There is a semantic relation between an object and a set of related objects (machine-infrastructure machines).
  • It is possible to identify a set of common API fields across all the related object implementations (status.infrastructureReady).

PROS…

  • It allows swappable implementations.
  • Extensively used in Cluster API.

CONS…

  • It requies all the stakeholders to agree on an API contract.
  • Evolving the contract requires coordination across all stakeholders.
  • It requires to use unstructured/generic client for accessing field defined in the API contract.

Extend Capabilities of a Single Controller

This set of patterns explore how developers can implement plug-in systems that can bring to a next level the number of use cases a single controller can support.

In Process Extensions

The controller runs plugins into its own process, each plugin extends a controller behaviour.

USE WHEN…

  • You want to limit plug-in side effects.
  • Binary compatibility is an issue.

PROS…

  • WebAssembly is a cool technology with growing adoption.

CONS…

  • We are still early WASM (specially WASI) stages.
  • Support for WebAssembly isn’t at the same level of maturity in all languages.
  • It doesn’t solve the plugin distribution problem.

Out Of Process Extensions

The controller watches for the webhook configurations pointing to plugin instances running across the network. The controller calls registered plugins.

USE WHEN…

  • Plug-ins require a dedicated security context.
  • It is requred to add/remove plugins at runtime.

PROS…

  • Plug in side-effects cannot impact the main controller.
  • same maintenance approach for plug in as the rest of the system.

CONS…

  • The controller has a dependency on external services.
  • It is required to manage authentication/authorization, network policies, failure policies, back pressure etc..
  • Organizational implications.

In this post, we learned that Design Patterns For Extensible Controllers. Thanks for Fabrizio Pandini & Rafael Fernandez Lopez for this session.

That’s all I’m going to talk about for now. See you in my next post!

References:

KubeCon + CloudNativeCon Europe 2022: Schedule

KubeCon + CloudNativeCon Europe 2022: Kubernetes is Your Platform: Design Patt… (sched.com)

Kubernetes Patterns (redhat.com)

--

--

--

KoçSistem’s software and technology stories

Recommended from Medium

Selector in CSS

output of above css

Mobile and Laptop push notifications over Python

Bankruptcy Attorney Salt Lake City

Python Lambda Function

Premium Lab: Buffer Overflow — Command Line Args

Open Ocean Council Website Launch

Hi, I have applied your solution couple of times with incorrect data, When now I have the correct…

Create AR business card using Unity

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sinem Merve Yılmaz

Sinem Merve Yılmaz

Software Quality Assurance Engineer

More from Medium

Different Types of Services in Kubernetes

Pod Disruption Budget — Budget that can save you One day :)

Kubernetes Architecture

Cloud Service Composition