CSI is not only about solving crime

Standardise that Storage API !

Every good fantasy story starts with “Once upon a time”.

So, voilà.

Once upon a time, there was a container orchestrator called Kubernetes.

Little PODs were swimming happily in this world.

They would come, disappear, and maybe come back to restart a cycle of life.

One day, a POD travelled with heavy luggage that it did not want to lose.

It heard there were many ways to store it, depending on its characteristics.

It is true that baggage could be big or small, cheap or expensive, important or meaningless.

A good opportunity to create a service to safely store it!

Suddenly, a whole bazaar was made available, each shop proposing its own way to safely store luggage.

Some shops would be blue.

Some shops would be green.

Some shops would be local.

Some shops would be free.

But there was no real consistency between them.

How would my little PODs choose wisely? It was all but easy.

Little PODs did not want to have to think about it.

A team of semi-gods was looking at this from their heavenly clouds.

After seeing this complex variety, they gathered, thought, argued, discussed, compromised and finally made a decision that was heard all over the world.

“Thou shall not be different,

Luggage shall be stored and accessed thru the concierge we call Sea Essaï!”

And just like that, life became easy again.

Little PODs could travel with luggage without effort!

And they lived happily ever after.

End of the story

Let’s put our feet back on solid ground.

Before 2018, there were already “in-tree” storage drivers within Kubernetes. Those were part of the upstream code. If a storage vendor wanted to be part of this, he had to follow the Kubernetes release cadence & processes, not to mention security issues & head count to test added code.

CSI (Container Storage Interface) was created to add an intermediate level of standardized API that allows container orchestrators (such as Kubernetes, Nomad, etc..) to provision block & file volumes.

With CSI, Storage vendors can propose storage drivers without modifying the core code of Kubernetes, which opens a lot of doors for many companies.

87 different storage systems were referenced in the first GA version of CSI drivers, on top of the 20 in-tree providers. That is a lot of storage!

The current complete list can be fetched on this link: https://kubernetes-csi.github.io/docs/drivers.html

This standard supports a variety of architecture, from a centralised management to a headless architecture. Also, CSI includes a set of sidecar containers, each one having a specific role.

“Sidecar, you said ” ?

You probably know sidecars as some kind of 3 wheeled motorbikes which carry an additional passenger. This passenger can be as useful as giving direction, or he can also just enjoy the view…

Does “Junior !” & “Professor Jones!” ring a bell?

Anyway, in Kubernetes, a sidecar is an extra container in a pod that brings a set of extra tasks to an application, without necessarily being part of that application. As an example, some data protection tools add sidecars to a pod to interact with the application when creating a backup.

Figure1: Example of a centralised deployment with one controller plugin & node plugins

There are currently 6 different sidecars:

• External-provisioner:

This is the CSI Driver API watcher! When someone creates a PVC (Persistent Volume Claim) against a specific Storage Class, which refers to a storage provisioner, this watcher will trigger a volume creation from this storage vendor.

Example of a Persistent Volume Claim that calls a specific Storage Class:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: blogpvc
spec: 
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
  storageClassName: storage-nas

Notice that there isn’t any reference to a storage feature (size is not a feature). The end user does not really need to know anything about storage. He is just a consumer after all.

Example of a Storage Class that calls for a CSI Driver provided by NetApp (https://github.com/netapp/trident):

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: storage-nas
provisioner: csi.trident.netapp.io
allowVolumeExpansion: true

This example is also quite easy to understand. Also, many extra parameters could be added to bring more features (thin provisioning, encryption, etc…).

• External-attacher:

One of the good things about this CSI standard, is that sidecars’ names are pretty self-explanatory… This piece is also quite important as it has the role of attaching/detaching volumes to/from nodes (or Publish/Unpublish to be more precise).

• External-snapshotter:

This sidecar is similar to the external-provisioner, but instead of watching for PVC creation, it is looking for VolumeSnapshot creation, which will lead to what is usually referred to as a “CSI Snapshot” or a point in time image of the content of a persistent volume.

A snapshot is created against two elements:

o The PVC it refers to

o The Volume Snapshot Class that calls for a specific CSI driver

In order to work, the Kubernetes admin must also install a set of objects that are not necessarily always present during the initial configuration:

o Kubernetes Volume Snapshot CRD

o Volume Snapshot Controller

More details about those: https://github.com/kubernetes-csi/external-snapshotter

Example of a Volume Snapshot that calls a specific Volume Snapshot Class:

kind: VolumeSnapshot
apiVersion: snapshot.storage.k8s.io/v1
metadata:
  name: blogsnapshot
spec: 
  volumeSnapshopClassName: snapshot-nas
  source:
    persistentVolumeClaimName: blogpvc

Example of a Volume Snapshot Class that calls for a CSI Driver (again provided by NetApp here):

kind: VolumeSnapshotClass
apiVersion: snapshot.storage.k8s.io/v1
metadata:
  name: snapshot-nas
driver: csi.trident.netapp.io

Note that you can create a new PVC (some kind of clone) from this VolumeSnapshot. To do so, you would need to add the following paragraph at the end of its manifest:

…
dataSource:
  name: blogsnapshot
  kind: VolumeSnapshot
  apiGroup: snapshot.storage.k8s.io/v1

You can see the following diagram the relationship between a PVC, a Volume Snapshot and a PVC created from this snapshots, 3 objects that are namespace bound, with their respective cluster wide resources, PV and VolumeSnapshotContent:

Figure2: Relationship between volumes and snapshots

• External-resizer:

This sidecar is in charge of watching persistent volumes resize requests, done through editing such resources or by patching it, and passes the order to the CSI driver to grow the volume.

This feature is enabled when the allowVolumeExpansion parameter is set in the storage class.

Note that volume shrinking is not possible at the time this article was written.

• Node-driver-registrar:

API communication (especially driver registration) between Kubelet and a CSI driver is done through this sidecar.

• Livenessprobe:

Finally, the CSI driver’s health is monitored by this sidecar. Whether there is an issue with it, Kubernetes will detect it & try to fix the problem.

Here you go, I hope you enjoyed this short introduction to the CSI specs.

Drop me a note to discuss more about Storage & Kubernetes !