Using Cinder as Storage Backend Kubernetes
Deploy Cinder Block Storage for Storage Kubernetes on top of OpenStack using CSI Driver
Intro
What is Cinder and Why ?
Cinder is an open-source project part of the OpenStack project that has a function for providing and Managing block storage in the OpenStack Cluster. Cinder uses API (Application Programming Interface) for users to interact with cinder, to create and use storage volume.
The OpenStack Block Storage service (Cinder) adds persistent storage to a virtual machine. Block Storage provides an infrastructure for managing volumes, and interacts with OpenStack Compute to provide volumes for instances. The service also enables management of volume snapshots, and volume types.
- OpenStack Docs
Component of Cinder:
- cinder-api
- cinder-volume
- cinder-scheduler daemon
- cinder-backup daemon
- Messaging queue
If you have Kubernetes deployment on top of OpenStack, you can use the cinder part of your OpenStack to provide a Storage Backend for Kubernetes without deploying any storage filesystem. Kubernetes can self-manage cinder to provision storage for your Kubernetes cluster itself with Cloud Provider OpenStack.
How ?
Kubernetes can communicate with OpenStack using Cloud Provider OpenStack. Cloud providers not only provide integration between Kubernetes and OpenStack for control cinder but many plugins are provided for integrating Kubernetes and OpenStack. But for now, we can discuss CSI Plugin for integration.
The Cinder CSI Driver is a CSI Specification compliant driver used by Container Orchestrators to manage the lifecycle of OpenStack Cinder Volumes.
Demo
in this demo, we’ll be using RKE (Rancher Kubernetes Engine) deployed on OpenStack Compute with sample service for testing using Mysql.
Prerequisites
- OpenStack RC file (We need to know auth OpenStack)
- Helm v3 already installed
Create Secret in Kubernetes for auth OpenStack
Create OpenStack RC files
vim cloud.conf
...
[Global]
username = YOUR_USER
password = YOUR_PASSWORD
domain-name = default
auth-url = https://YOUR_DU_URL/keystone/v3
tenant-id = YOUR_TENANT_ID
region = YOUR_REGION
Convert into base64 for value in Kubernetes secret
cat cloud.conf | base64 |tr -d '\n'
W0dsb2JhbF0KdXNlcm5hbWUgPSBZT1VSX1VTRVIKcGFzc3dvcmQgPSBZT1VSX1BBU1NXT1JECmRvbWFpbi1uYW1lID0gZGVmYXVsdAphdXRoLXVybCA9IGh0dHBzOi8vWU9VUl9EVV9VUkwva2V5c3RvbmUvdjMKdGVuYW50LWlkID0gWU9VUl9URU5BTlRfSUQKcmVnaW9uID0gWU9VUl9SRUdJT94K
Create Kubernetes secret and store OpenStack RC
vim csi-secret-cinderplugin.yaml
kind: Secret
apiVersion: v1
metadata:
name: cloud-config
namespace: kube-system
data:
cloud.conf: W0dsb2JhbF0KdXNlcm5hbWUgPSBZT1VSX1VTRVIKcGFzc3dvcmQgPSBZT1VSX1BBU1NXT1JECmRvbWFpbi1uYW1lID0gZGVmYXVsdAphdXRoLXVybCA9IGh0dHBzOi8vWU9VUl9EVV9VUkwva2V5c3RvbmUvdjMKdGVuYW50LWlkID0gWU9VUl9URU5BTlRfSUQKcmVnaW9uID0gWU9VUl9SRUdJT04K
Apply
kubectl create -f csi-secret-cinderplugin.yaml
Deploy the CSI-Cinder Controller and Plugins
First, pull CSI-Cinder from GitHub
git clone https://github.com/kubernetes/cloud-provider-openstack.git
cd cloud-provider-openstack/
Deploy CSI-Cinder
kubectl apply -f manifests/cinder-csi-plugin/
Validate Installation
kubectl get pods -n kube-system
...
csi-cinder-controllerplugin-0 6/6 Running 6 8d
csi-cinder-nodeplugin-4w6w6 3/3 Running 3 8d
csi-cinder-nodeplugin-gk5nf 3/3 Running 3 8d
Create Cinder as a Storage Class
Create StorageClass file
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: cinder
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: cinder.csi.openstack.org
Apply
kubectl create -f storage-class.yaml
Validate
kubectl get sc
...
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
cinder (default) cinder.csi.openstack.org Delete Immediate false 26m
Create App For testing
Create namespace
kubectl create ns app1
Pull Mysql helm chart
helm repo add bitnami https://charts.bitnami.com/bitnami
helm pull bitnami/mysql --untar
Edit values.yaml from Mysql helm chart for using Cinder as StorageClass
persistence:
enabled: true
storageClass: "cinder"
Deploy Mysql
helm install -f values.yaml mysql my-repo/mysql -n app1
...
NAME: mysql
LAST DEPLOYED: Thu Nov 17 01:48:06 2021
NAMESPACE: app1
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.4.3
APP VERSION: 8.0.31
Check PVC
kubectl get pvc -n app1
...
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-mysql-0 Bound pvc-2885d8cd-b6e4-40ad-bedf-68bf303097ca 8Gi RWO cinder 14m
Check Database
kubectl run mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:8.0.31-debian-11-r10 --namespace app1 --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command -- bash
I have no name!@mysql-client:/$ mysql -h mysql.app1.svc.cluster.local -uroot -p"P@ssw0rd"
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| my_database |
| mysql |
| performance_schema |
| sys |
+--------------------+
5 rows in set (0.00 sec)
mysql> exit
You can see in OpenStack dashboard
Closing
With CSI-Cinder plugin we can easily to provision Storage Backend for our Kubernetes using Cinder. We can use cinder in the same OpenStack or different, we just need OpenStack RC files for auth and Cloud Provider OpenStack for integration.
By Fauzan Rafi, Research Team Btech
Our Tagline
#ContinuousLearning
Reference
https://docs.openstack.org/cinder/rocky/install/get-started-block-storage.html