Using Cinder as Storage Backend Kubernetes

Deploy Cinder Block Storage for Storage Kubernetes on top of OpenStack using CSI Driver

Kubernetes on top of OpenStack

Intro

What is Cinder and Why ?

Cinder is an open-source project part of the OpenStack project that has a function for providing and Managing block storage in the OpenStack Cluster. Cinder uses API (Application Programming Interface) for users to interact with cinder, to create and use storage volume.

The OpenStack Block Storage service (Cinder) adds persistent storage to a virtual machine. Block Storage provides an infrastructure for managing volumes, and interacts with OpenStack Compute to provide volumes for instances. The service also enables management of volume snapshots, and volume types.
- OpenStack Docs

Component of Cinder:

• cinder-api
• cinder-volume
• cinder-scheduler daemon
• cinder-backup daemon
• Messaging queue

If you have Kubernetes deployment on top of OpenStack, you can use the cinder part of your OpenStack to provide a Storage Backend for Kubernetes without deploying any storage filesystem. Kubernetes can self-manage cinder to provision storage for your Kubernetes cluster itself with Cloud Provider OpenStack.

How ?

Kubernetes can communicate with OpenStack using Cloud Provider OpenStack. Cloud providers not only provide integration between Kubernetes and OpenStack for control cinder but many plugins are provided for integrating Kubernetes and OpenStack. But for now, we can discuss CSI Plugin for integration.

The Cinder CSI Driver is a CSI Specification compliant driver used by Container Orchestrators to manage the lifecycle of OpenStack Cinder Volumes.

Demo

in this demo, we’ll be using RKE (Rancher Kubernetes Engine) deployed on OpenStack Compute with sample service for testing using Mysql.

Prerequisites

• OpenStack RC file (We need to know auth OpenStack)
• Helm v3 already installed

MySQL

Create Secret in Kubernetes for auth OpenStack

Create OpenStack RC files

vim cloud.conf
...
[Global]
username = YOUR_USER
password = YOUR_PASSWORD
domain-name = default
auth-url = https://YOUR_DU_URL/keystone/v3
tenant-id = YOUR_TENANT_ID
region = YOUR_REGION

Convert into base64 for value in Kubernetes secret

cat cloud.conf | base64 |tr -d '\n'

W0dsb2JhbF0KdXNlcm5hbWUgPSBZT1VSX1VTRVIKcGFzc3dvcmQgPSBZT1VSX1BBU1NXT1JECmRvbWFpbi1uYW1lID0gZGVmYXVsdAphdXRoLXVybCA9IGh0dHBzOi8vWU9VUl9EVV9VUkwva2V5c3RvbmUvdjMKdGVuYW50LWlkID0gWU9VUl9URU5BTlRfSUQKcmVnaW9uID0gWU9VUl9SRUdJT94K

Create Kubernetes secret and store OpenStack RC

vim csi-secret-cinderplugin.yaml

kind: Secret
apiVersion: v1
metadata:
  name: cloud-config
  namespace: kube-system
data:
  cloud.conf: W0dsb2JhbF0KdXNlcm5hbWUgPSBZT1VSX1VTRVIKcGFzc3dvcmQgPSBZT1VSX1BBU1NXT1JECmRvbWFpbi1uYW1lID0gZGVmYXVsdAphdXRoLXVybCA9IGh0dHBzOi8vWU9VUl9EVV9VUkwva2V5c3RvbmUvdjMKdGVuYW50LWlkID0gWU9VUl9URU5BTlRfSUQKcmVnaW9uID0gWU9VUl9SRUdJT04K

Apply

kubectl create -f csi-secret-cinderplugin.yaml

Deploy the CSI-Cinder Controller and Plugins

First, pull CSI-Cinder from GitHub

git clone https://github.com/kubernetes/cloud-provider-openstack.git
cd cloud-provider-openstack/

Deploy CSI-Cinder

kubectl apply -f manifests/cinder-csi-plugin/

Validate Installation

kubectl get pods -n kube-system
...
csi-cinder-controllerplugin-0             6/6     Running     6          8d
csi-cinder-nodeplugin-4w6w6               3/3     Running     3          8d
csi-cinder-nodeplugin-gk5nf               3/3     Running     3          8d

Create Cinder as a Storage Class

Create StorageClass file

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: cinder
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: cinder.csi.openstack.org

Apply

kubectl create -f storage-class.yaml

Validate

kubectl get sc
...
NAME               PROVISIONER                RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
cinder (default)   cinder.csi.openstack.org   Delete          Immediate           false                  26m

Create App For testing

Create namespace

kubectl create ns app1

Pull Mysql helm chart

helm repo add bitnami https://charts.bitnami.com/bitnami 
helm pull bitnami/mysql --untar

Edit values.yaml from Mysql helm chart for using Cinder as StorageClass

  persistence:
    enabled: true
    storageClass: "cinder"

Deploy Mysql

helm install -f values.yaml mysql my-repo/mysql -n app1
...
NAME: mysql
LAST DEPLOYED: Thu Nov 17 01:48:06 2021
NAMESPACE: app1
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.4.3
APP VERSION: 8.0.31

Check PVC

kubectl get pvc -n app1
...
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-mysql-0   Bound    pvc-2885d8cd-b6e4-40ad-bedf-68bf303097ca   8Gi        RWO            cinder         14m

Check Database

kubectl run mysql-client --rm --tty -i --restart='Never' --image  docker.io/bitnami/mysql:8.0.31-debian-11-r10 --namespace app1 --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command -- bash

I have no name!@mysql-client:/$ mysql -h mysql.app1.svc.cluster.local -uroot -p"P@ssw0rd"

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| my_database        |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

mysql> exit

You can see in OpenStack dashboard

Closing

With CSI-Cinder plugin we can easily to provision Storage Backend for our Kubernetes using Cinder. We can use cinder in the same OpenStack or different, we just need OpenStack RC files for auth and Cloud Provider OpenStack for integration.

By Fauzan Rafi, Research Team Btech

Our Tagline

#ContinuousLearning

Reference

https://docs.openstack.org/cinder/rocky/install/get-started-block-storage.html

How to Deploy a Cinder Storage Backend for your Kubernetes Cluster using CSI - Platform9 Learning

cloud-provider-openstack/using-cinder-csi-plugin.md at master · kubernetes/cloud-provider-openstack