Understanding the Container Storage Interface (CSI)
Introduction
Container Storage Interface (CSI) is an initiative to unify the storage interface of Container Orchestrator Systems (COs) like Kubernetes, Mesos, Docker swarm, cloud foundry, etc. combined with storage vendors like Ceph, Portworx, NetApp etc. This means, implementing a single CSI for a storage vendor is guaranteed to work with all COs.
Note that, we discuss only about dynamic provisioning in this article. Pre-provisioned volumes and flex volumes are out of scope of this article. If you do know what I am talking about, read more about them here. Also, this article is not going to deep-dive into the implementation details of CSI. This document gives a good high-level overview and is a good prerequisite to start implementing CSI. Also, examples and references are based on Kubernetes COs.
Before we dive in, one should know what sidecar containers are in Kubernetes. Sidecar containers extend and enhance the “main” container. They exist in the same pod as the “main” container sharing storage and network with it.
At the time of writing the following are version status of the CSI components.
Before CSI
The first release of CSI v0.1 was in December 2017. It was ofcourse possible to provision external storage in COs before CSI existed. Volume plugins were serving the storage needs for container workloads in case of Kubernetes.
The volume plugins are part of the COs core as shown in the picture above. Due to this, it has the following drawbacks mentioned in the CSI design document:
- Volume plugin development is tightly coupled and dependent on Kubernetes releases.
- Kubernetes developers/community are responsible for testing and maintaining all volume plugins, instead of just testing and maintaining a stable plugin API.
- Bugs in volume plugins can crash critical Kubernetes components, instead of just the plugin.
- Volume plugins get full privileges of kubernetes components (kubelet and kube-controller-manager).
- Plugin developers are forced to make plugin source code available, and can not choose to release just a binary.
Understanding CSI
On introduction of CSI, Kubernetes team released some external components which are not part of the core and that can interact with vendor implemented external components. They communicate to each other over gRPC on domain sockets.
Kubernetes external component
This is completely implemented and maintained by the Kubernetes team. These extend kubernetes actions outside of kubernetes. The vendors need not worry about the implementation details of this at all. They consists of three sub-components.
- Driver registrar — is a sidecar container that registers the CSI driver with kubelet, and adds the drivers custom NodeId to a label on the Kubernetes Node API Object. It does this by communicating with the Identity service on the CSI driver and also calling the CSI GetNodeId operation.
- External provisioner — is a sidecar container that watches Kubernetes PersistentVolumeClaim objects and triggers CSI CreateVolume and DeleteVolume operations against a driver endpoint.
- External attacher — is a sidecar container that watches Kubernetes VolumeAttachment objects and triggers CSI ControllerPublish and ControllerUnpublish operations against a driver endpoint
Storage vendor/3rd-party external component
This is a vendor specific implementation. Each vendor should implement their respective APIs into gRPC service functions. E.g. Implementation of GCE PD, Ceph, etc. They too consists of three sub-components.
- CSI Identity — is mainly for identifying the plugin service, making sure it’s healthy, and returning basic information about the plugin itself.
- CSI Controller — is responsible of controlling and managing the volumes, such as: creating, deleting, attaching/detaching, snapshotting, etc.
- CSI Node — is responsible for controllong volume’s action in the kubernetes node.
Conclusion
With the introduction of CSI, there is a clear benefit for the COs and storage vendors. Due to its well-defined interfaces, it also helps developers and future COs to easily implement and test CSI. At this point, if you decide to start implementing you own CSI, this is a good place to start with: how-to implement a CSI from FatihArlsan.
References
- CSI spec: https://github.com/container-storage-interface/spec/blob/master/spec.md
- Kubernetes sidecar containers: https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns/
- KubeCon EU: CloudNativeCon EU 2018 CSI Jie Yu
- CSI design document: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md
- CSI docs: https://kubernetes-csi.github.io/docs/
- CSI docs (deprecated): https://github.com/kubernetes-csi/docs/wiki/Usage
