How To Write Validating and Mutating Admission Controller Webhooks in Python for Kubernetes

Photo by Mattia Serrani on Unsplash

Admission Controller Webhooks are a great way to control what is being deployed to a Kubernetes cluster. In this story we will cover how to write both using the Python web framework Flask. It is assumed that readers have some basic knowledge on how resources work in Kubernetes and how to deploy a Flask applications to a Kubernetes cluster.

What does an Admission Controller Webhook do?

An Admission Controller Webhook is triggered when a Kubernetes resource (or resources) is created, modified or deleted. Essentially a HTTP request is sent to a specified Kubernetes Service in a namespace which returns a JSON response. Depending on that response an action is taken.

There are two categories of Admission Controllers, Validating and Mutating. A Validating Admission Controller validates the incoming request and returns a binary response, yes or no based on custom logic. An example can be that if a Pod resource doesn’t have certain labels the request is rejected with a message on why. A Mutating Admission Controller modifies the incoming request based on custom logic. An example can be that if an Ingress resource doesn’t have the correct annotations, the correct annotations will be added and the resource will be admitted.

With the above scenarios Admission Controllers can be very powerful and give very granular control over what goes in or out of a Kubernetes Cluster. Now let’s dive in.

Note: The code examples shown below have been tested with Kubernetes version 1.14.2. Please read the documentation for more recent versions.

Writing a Validating Admission Controller Webhook

Before we begin lets create a specification:

• We are going to validate a Deployment resource to check it has a label. If the label is not present then we reject the request.
• The above will happen when a new Deployment is created.

validating_admission_controller.py

from flask import Flask, request, jsonify
admission_controller = Flask(__name__)
@admission_controller.route('/validate/deployments', methods=['POST'])
def deployment_webhook():
    request_info = request.get_json()
    if request_info["request"]["object"]["metadata"]["labels"].get("allow"):
        return admission_response(True, "Allow label exists")
    return admission_response(False, "Not allowed without allow label")
def admission_response(allowed, message):
    return jsonify({"response": {"allowed": allowed, "status": {"message": message}}})
if __name__ == '__main__':
    admission_controller.run(host='0.0.0.0', port=443, ssl_context=("/server.crt", "/server.key"))

In the above, we write a function which does the following:

• Fetches the incoming request object. In our case we can assume the object will always be a Deployment resource when we register our Validating Controller below.
• Check if the “allow” label exists. Return a JSON HTTP response if present with the allowed boolean set to True and a message.
• If the above condition is not met we will always reject any Deployment resources from being created.

The above can now be deployed. We will not go into the details on how to deploy the above but I’ll give a quick list of bullet points:

• Create a Docker image from admission_controller.py with Flask installed.
• Generate a self signed CA, generate a csr and cert then create a secret based on this cert.
• Create a Deployment from the created Docker image in a namespace. The service must be secured via SSL. Mount the secret created from the previous step as volumes in the Deployment.
• Create a Service pointing to the correct ports in same namespace as the Deployment.

To register our Validating Controller we need to apply the following configuration:

validating_admission_webhook.yaml

apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  name: validating-webhook
  namespace: test
webhooks:
  - name: test.example.com
    failurePolicy: Fail
    clientConfig:
      service:
        name: test-validations
        namespace: test
        path: /validate/deployments
      caBundle: <redacted> # a base64 encoded self signed ca cert is needed because all Admission Webhooks need to be on SSL
    rules:
      - apiGroups: ["apps"]
        resources:
          - "deployments"
        apiVersions:
          - "*"
        operations:
          - CREATE

In the above example we can see that we are only interested in deployments, as we narrow down on which resources the web hook will execute. A self-signed SSL cert is required with the CN(in our case) deployment-test.test.svc.

We can apply the above by running:

kubectl apply -f validating_admission_webhook.yaml

To validate, create a new Deployment with the “allow” label in the metadata section with any value and it should be created. For example, we can try to deploy nginx.

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1
        ports:
        - name: http
          containerPort: 80

The above will fail with the following message:

Error from server: error when creating "nginx.yaml": admission webhook "test.example.com" denied the request: Not allowed without allow label

Try adding the allow label and see if it passes. Please reach out in the comments section if you need assistance.

Writing a Mutating Admission Controller Webhook

Before we begin lets create a specification:

• We are going to mutate a Deployment resource and a label to it.
• The above will happen when a new Deployment is created.

mutating_admission_controller.py

from flask import Flask, request, jsonify
import base64
import jsonpatch
admission_controller = Flask(__name__)
@admission_controller.route('/mutate/deployments', methods=['POST'])
def deployment_webhook_mutate():
    request_info = request.get_json()
    return admission_response_patch(True, "Adding allow label", json_patch = jsonpatch.JsonPatch([{"op": "add", "path": "/metadata/labels/allow", "value": "yes"}]))
def admission_response_patch(allowed, message, json_patch):
    base64_patch = base64.b64encode(json_patch.to_string().encode("utf-8")).decode("utf-8")
    return jsonify({"response": {"allowed": allowed,
                                 "status": {"message": message},
                                 "patchType": "JSONPatch",
                                 "patch": base64_patch}})
if __name__ == '__main__':
    admission_controller.run(host='0.0.0.0', port=443, ssl_context=("/server.crt", "/server.key"))

In the above, we write a function which does the following:

• Fetches the incoming request object. In our case we can assume the object will always be a Deployment resource when we register our Mutating Controller below.
• Add an allow label with value “yes” and return the base64 encoded JSONPatch in the response.

The deployment details are similar to the Validating Admission controller, please refer to previous section. To register our mutating controller we need to create the following.

mutating_admission_webhook.yaml

apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  name: mutating-webhook
  namespace: test
  labels:
    component: mutating-controller
webhooks:
  - name: test.example.com
    failurePolicy: Fail
    clientConfig:
      service:
        name: test-mutations
        namespace: test
        path: /mutate/deployments
      caBundle: <redacted> # a base64 encoded self signed ca cert is needed because all Admission Webhooks need to be on SSL
    rules:
      - apiGroups: ["apps"]
        resources:
          - "deployments"
        apiVersions:
          - "*"
        operations:
          - CREATE

In the above example we can see again that we are only interested in deployments. A self-signed SSL cert is required with the CN(in our case) test-mutations.test.svc.

We can apply the above by running:

kubectl apply -f mutating_admission_webhook.yaml

To validate, create a new Deployment and the allow label will be added to the new Deployment:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  ...
  labels:
    allow: "yes"
    app: nginx
  name: nginx
  ...

Thank you for reading, I hope this was useful. Please reply in the comments section if you run into any issues.