Migrate PVC from one storage class to another
There could be a requirement where you might need to migrate the PVC and its data from one storage class (storage back-end) to another. This could be because of migration from one storage solution to another due to cost and performance factor.
There are two ways we can migrate the PVC from one storage class to another.
- Cloning PVC (volume) from one storage class to another using csi-driver
- Migrate PVC using
korb
tool
In this article we will see both of the ways one by one.
Contents
- Why it works on my machine ?
- Clone PVC using CSI-driver
- Migrate PVC using Korb tool
Why it works on my machine?
I am using:
- Kubernetes version 1.28.7
- One control plane and two worker nodes on Ubuntu 22.04
- Containerd CRI 1.6.28
- csi-driver-nfs v4.5.0 as csi-driver for nfs with external snapshotter enabled
- korb version 2.2.0
Clone PVC using csi-driver
PVC cloning feature is only supported for csi-based PVCs using CSI drivers.
A Clone is a duplicate of an existing Kubernetes Volume that can be consumed as any standard Volume would be. The only difference is that upon provisioning, rather than creating a new empty Volume, the back end device creates an exact duplicate of the specified Volume.
Below requirements must be met to clone a PVC:
- CSI driver of storage provisioner must support cloning feature. Please check with your csi-driver provider to know if PVC cloning feature is supported.
- Cloning can only be performed between same volumemode storage. For e.g. source volume mode is filesystem, cloned PVC must also have filesystem volume mode.
- Make sure csi external snapshotter is deployed on your cluster. Check with your csi-driver provider to deploy it on cluster.
- The source PVC must be bound to a PV.
For this demo, I have two storage classes source nfs-csi
and target new-nfs-csi
. Both are backed by NFS servers running on two different servers.
Let’s start our demo:
- Create storage class for target storage provisioner. I have target storage class with name
new-nfs-sc
.
2. I already have a PVC deployed on my cluster with source storage class and mounted to a pod.
3. Let me copy a 1G of file inside PVC using kubectl cp
command for testing.
I have nginx pod on which source PVC is mounted at /var/log/nginx path.
4. Create PVC clone using below yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nginx-logs-clone # <------------- Cloned PVC name
spec:
storageClassName: new-nfs-sc # <------- Target storage class
dataSource: # <------------------------ This field provides the source of PVC
name: nginx-logs # <------------------ Source PVC name
kind: PersistentVolumeClaim
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
As you can see in above YAML, there is a dataSource
field where we can mention source PVC details. Kubernetes and CSI-driver uses that field to identify the source PVC which need to be cloned. storageClassName
field has storage class name of target storage class.
The cloned PVC size must not be less than size of source PVC.
Once we apply above YAML file, We will see that a new PVC is created. It will create a PV in target storage class and copy the content of source PV to cloned PV.
5. Now lets check the content of cloned PV.
A. To check the content, We will mount the cloned PV as well to same pod by editing the nginx deployment.
B. Compare the content between source PVC and cloned PVC
Now that you see PVC can be cloned using csi-driver. There are some limitations on this. Some of the limitations are:
- This way is only possible with csi-backed PVCs
- CSI-driver provided by storage provisioner must support volume cloning feature
- We need exxternal snapshotter to be deployed on cluster
To know more about volume cloning, Follow Kubernetes document — https://kubernetes.io/docs/concepts/storage/volume-pvc-datasource/
Migrate PVC using korb tool
Korb is a open source tool which is used to migrate a PVC from one storage class to another. This works with any type of storage class and need not be csi-backed.
It internally uses rsync and copies the content from PVC on one storage class to another and then deletes the source PVC.
To use Korb
we must download Korb tool on jumpbox or VM from where we can connect cluster.
curl -LO https://github.com/BeryJu/korb/releases/download/v2.1.3/korb_2.1.3_linux_amd64.tar.gz
tar -xvzf korb_2.1.3_linux_amd64.tar.gz
sudo mv korb /usr/local/bin/korb
Currently I have PVC in nfs-csi storage class. I willl migrate it to new-nfs-csi storage class.
Before cloning a PVC, Pods using the PVC must be scaled down.
Now, To clone the PVC, use below command
korb --new-pvc-storage-class=<name-of-target-storage-class> --strategy=copy-twice-name <source-pvc-name>
Above command will create the PVC with same name as source PVC but in different storage class with same content. It may take time to copy data from source to migrated PVC depending on data size.
If you want to create PVC with different name, --new-pvc-name
flag can be used with new pvc name argument.
Lets check the new PVC. Now the PVC is created with same name but in another storage class new-nfs-sc.
Now let’s scale up nginx pod and check the content inside PVC. Since PVC name is still same, I don’t need to edit deployment to mount new PVC.
As you can see the content is retained in new PVC with new storage class.
You can read more about Korb tool here — https://github.com/BeryJu/korb
I hope this article must have given you idea about migration of PVCs from one storage class to another.
You can support my work by buying me a cup of coffee on https://www.buymeacoffee.com/linuxshots
Thank You!
Navratan Lal Gupta
Linux Shots