Building stuff with the Kubernetes API (Part 3) — Using Python

Published in

Programming Kubernetes

7 min readMar 17, 2018

This is part 3, of a multipart series, where I walkthrough the different options for working with the Kubernetes API. Here we will cover one more OpenAPI-based client, mainly, the official Kubernetes Python client framework.

The Kubernetes Python Client

The Kubernetes Python client framework is an OpenAPI client, which means it uses a Swagger code generator (https://github.com/kubernetes-client/gen) to generate OpenAPI-compliant serializers to access core and extended API objects.

As of this writing, the Python client is split in two repos: python-base, which contains utility types and functions, and the python repo which houses the generated OpenAPI Kubernetes objects, the generated serializers, and the HTTP transport and configuration.

A simple client tool for Kubernetes

Let us review the Kubernetes client tool we are going to build to illustrate the usage of the Python client framework. pvcwatch, is a CLI tool which watches the total claimed persistent storage capacity in a cluster. When the total reaches a threshold, it takes an action (in this example, it’s a simple notification on the screen).

You can find the complete example on GitHub.

The example is designed to highlight several aspects of Kubernetes Python client including:

connectivity
resource list retrieval and walk through
object watch

Setup

A note on setting up the Python packages is in order. I find that using the pip manager works well for setting up the Kubernetes Python packages using pip install kubernetes.

Tip: if running OXS High Sierra, you may need to use pip install --user kubernetes when installing the Python packages to avoid permission issues.

Also, the Python client does not include its own unit and quantity classes as was with Java (and Go). Fortunately, I found one called pint that works well for representing volume quantities. Make sure to get it using pip install pint.

Connecting to the API server

The first step in our Python client program will be to connect to the API server. To do this, we use the config package as shown.

...
from kubernetes import client, config, watchdef main():
    config.load_kube_config()

Package config (found in project python-base) provides utility functions to help setup connectivity to an API server.

From a config file

If you are building a stand-alone tool that has access to a kubeconfig file, use the following function. When no parameters are provided, the function defaults to loading the kubeconfig file from $HOME/.kube/config.

config.load_kube_config()

Within a cluster

If your code will be deployed in a Kubernetes cluster, use the following to configure the connection using cluster information.

config.load_incluster_config()

Create an API object serializer

Next we need to create a serializer to process typed API objects to and from the server. The Swagger-generated types in the client package include versioned API object classes and serializer classes which provide access to API operations.

For our example, we instantiate class CoreV1Api to access V1 version of Kubernetes core API objects as shown.

...
from kubernetes import client, config, watchdef main():
    config.load_kube_config()
    api = client.CoreV1Api()

You can read about the versioned API objects and their serializers here.

Listing cluster PVCs

One of the most basic operations we can do with the API serializers is to retrieve resource lists of stored API objects. For our example, we are going to retrieve a namespaced list of PVCs as follows using method list_namespaced_persistent_volume_claim().

...
from kubernetes import client, config, watchdef main():    
    ns = os.getenv("K8S_NAMESPACE")
    if ns is None:
        ns = ""    config.load_kube_config()
    api = client.CoreV1Api()    pvcs = api.list_namespaced_persistent_volume_claim(
      namespace=ns, watch=False)

The list method above returns an API object of type V1PersistentVolumeClaimList which we can use to print PVC information as shown in the following snippet:

...
from kubernetes import client, config, watchdef main():    
    ...    pvcs = api.list_namespaced_persistent_volume_claim(
      namespace=ns, watch=False)    print("---- PVCs ---")
    print("%-16s\t%-40s\t%-6s" % ("Name", "Volume", "Size"))
    for pvc in pvcs.items:
        print("%-16s\t%-40s\t%-6s" %
              (pvc.metadata.name, pvc.spec.volume_name,    
               pvc.spec.resources.requests['storage']))

Watching the cluster PVCs

The Kubernetes Python client framework supports the ability to watch a cluster for API object events including ADDED, MODIFIED, DELETED generated when an object is created, updated, and removed respectively. For our simple CLI tool, we will use this watch capability to monitor the total capacity of claimed persistent storage against a running cluster.

When the total claim capacity, for a given namespace, reaches a certain threshold, we will take an arbitrary action. For simplicity sake, we will just print a notification on the screen. However, in a more sophisticated implementation, the same approach can be used to trigger a some automated action.

Streaming PVC events

Next we use the watch package to create a new Watch object. Then, loop through a stream of events returned by method watch.stream. Before we setup the watch, notice that we are using package pint to express the threshold for the maximum claim, max_claims, as a unit of storage quantity of 150Gi.

...
import pint
from kubernetes import client, config, watchdef main():
    ...
    unit = pint.UnitRegistry()
    unit.define('gibi = 2**30 = Gi')
    max_claims = unit.Quantity("150Gi")
    total_claims = unit.Quantity("0Gi")
    ...
    w = watch.Watch()
    for item in w.stream(
        api.list_namespaced_persistent_volume_claim, namespace=ns,
        timeout_seconds=0):
    ...

Notice that we have to set a timeout value, timout_seconds, for the stream. By setting it to 0, it forces the HTTP client to keep a long running connection to the server, avoiding timing out before the program receives any event.

Processing PVC added event

Inside the loop, we setup if-statements to inspect each watch item retrieved from the server. If the event type is “ADDED” , it means a new PVC has been created.

The following code extract the PVC, and its quantity, from the watch item. Then it updates the running total claim quantity, total_claims. If that total is greater than the threshold quantity, max_claims, a screen notification action is printed.

...
def main():
    ...
    max_claims = unit.Quantity("150Gi")
    total_claims = unit.Quantity("0Gi")
    ...
    w = watch.Watch()
    for item in w.stream(
        api.list_namespaced_persistent_volume_claim, namespace=ns,
        timeout_seconds=0):        pvc = item['object']        if item['type'] == 'ADDED':
            size = pvc.spec.resources.requests['storage']
            claimQty = unit.Quantity(size)
            total_claims = total_claims + claimQty            if total_claims >= max_claims:
                print("**** Trigger over capacity action ***")
         ...

Processing PVC deleted events

The code also monitors when PVCs are removed. It applies the reverse logic and decreases the running total claims when the PVC is deleted.

...
def main():
    ...
    max_claims = unit.Quantity("150Gi")
    total_claims = unit.Quantity("0Gi")
    ...
    w = watch.Watch()
    for item in w.stream(
        api.list_namespaced_persistent_volume_claim, namespace=ns,
        timeout_seconds=0):        pvc = item['object']
        ...
        if item['type'] == 'DELETED':
            size = pvc.spec.resources.requests['storage']
            claimQty = unit.Quantity(size)
            total_claims = total_claims - claimQty            if total_claims <= max_claims:
                print("INFO: claim usage normal; max %s; at %s" % 
                (max_claims, total_claims))
        ...

Run the program

Now let us run the program against a running cluster. It first displays the list of existing PVCs. Then it starts watching the cluster for new PVC events.

$> python pvc_watch.py---- PVCs ---
Name               Volume                                   Size
my-influx-influxdb pvc-59f760cd-1133-11e8-b7b8-08002739cded 75GiPVC Added: my-influx-influxdb; size 75Gi

Next, let us deploy another application unto the cluster that requests an additional 100Gi in storage claim (as we did before, let us use Helm to deploy a redis chart) while pvcwatch tool is still running.

helm install --name my-redis \
  --set persistence.enabled=true,persistence.size=100Gi stable/redis

As you can see below, pvcwatch immediately reacts to the new claim and displays the alert since we have gone over the threshold.

$> python pvc_watch.py---- PVCs ---
Name               Volume                                   Size
my-influx-influxdb pvc-59f760cd-1133-11e8-b7b8-08002739cded 75Gi
my-redis-redis     pvc-57f0b679-2948-11e8-ad9c-08002739cded 100GiPVC Added: my-influx-influxdb; size 75Gi
INFO: total PVC at 50.0% capacity
PVC Added: my-redis-redis; size 100Gi
---------------------------------------------
WARNING: claim overage reached; max 150 gibi; at 175 gibi
**** Trigger over capacity action ***
---------------------------------------------
INFO: total PVC at 116.7% capacity

Conversely, when a PVC is deleted from the cluster, the tool react accordingly with an alert message. Now, let us remove the redis application which will delete its PVC claim of 100Gi. As you can see below, the tool immediately reacts as expected.

> python pvc_watch.py---- PVCs ---
Name               Volume                                   Size
my-influx-influxdb pvc-59f760cd-1133-11e8-b7b8-08002739cded 75Gi
my-redis-redis     pvc-57f0b679-2948-11e8-ad9c-08002739cded 100Gi
...
INFO: total PVC at 116.7% capacity
PVC Deleted: my-redis-redis; size 100Gi
---------------------------------------------
INFO: claim usage normal; max 150 gibi; at 75 gibi
---------------------------------------------
INFO: total PVC at 50.0% capacity

Summary

In the last post, we started exploring how to build Kubernetes client tools using the Kubernetes Java client. This post did the same by showing how to create a simple PVC watching tool, this time however, using the Kubernetes Python client framework. We saw how to setup connection to an API server, retrieve object lists and configure a watch object to stream API events from the server.

What’s next

In the next post, we will start with the official Kubernetes Go client framework and explore similar concerns when building client programs that interact with the cluster.

If you find this writeup useful, please let me know by clicking on the clapping hands 👏 icon to recommend this post.