Velero with CSI
Published in
3 min readSep 7, 2020
- Pre-requistie Kubernetes Cluster > v1.7
2. Install the K8s SnapShot CRD’s
https://github.com/kubernetes-csi/external-snapshotter/issues/245https://github.com/kubernetes-csi/external-snapshotter/blob/master/README.md#usage--
root@k8s-storage-1:~# git clone https://github.com/kubernetes-csi/external-snapshotter.git
root@k8s-storage-1:~# cd external-snapshotter/
--
root@k8s-storage-1:~/external-snapshotter# kubectl create -f client/config/crd
customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created
customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created
customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created
--root@k8s-storage-1:~/external-snapshotter# kubectl create -f deploy/kubernetes/csi-snapshotter
serviceaccount/csi-snapshotter created
clusterrole.rbac.authorization.k8s.io/external-snapshotter-runner created
clusterrolebinding.rbac.authorization.k8s.io/csi-snapshotter-role created
role.rbac.authorization.k8s.io/external-snapshotter-leaderelection created
rolebinding.rbac.authorization.k8s.io/external-snapshotter-leaderelection created
serviceaccount/csi-provisioner created
clusterrole.rbac.authorization.k8s.io/external-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/csi-provisioner-role created
role.rbac.authorization.k8s.io/external-provisioner-cfg created
rolebinding.rbac.authorization.k8s.io/csi-provisioner-role-cfg created
clusterrolebinding.rbac.authorization.k8s.io/csi-snapshotter-provisioner-role created
rolebinding.rbac.authorization.k8s.io/csi-snapshotter-provisioner-role-cfg created
service/csi-snapshotter created
statefulset.apps/csi-snapshotter created
3. If you want to use CSI Snapshotting feature, you need to install the VolumeSnapShotClass and VolumeSnapshot also (from Rook)
kubectl apply -f https://raw.githubusercontent.com/rook/rook/master/cluster/examples/kubernetes/ceph/csi/rbd/snapshotclass.yamlvolumesnapshotclass.snapshot.storage.k8s.io/csi-rbdplugin-snapclass createdkubectl apply -f https://raw.githubusercontent.com/rook/rook/master/cluster/examples/kubernetes/ceph/csi/rbd/snapshot.yamlkubectl get VolumeSnapshotClass --all-namespaces
NAME DRIVER DELETIONPOLICY AGE
csi-rbdplugin-snapclass rook-ceph.rbd.csi.ceph.com Delete 67s
4. Installation Velero with CSI Enabled
velero install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.0.0,velero/velero-plugin-for-csi:v0.1.1 \
--bucket velero2 \
--secret-file ./credentials-velero \
--use-volume-snapshots=true \
--backup-location-config region=default,s3ForcePathStyle="true",s3Url=http://minio.10.131.232.223.nip.io \
--image velero/velero:v1.4.0 \
--snapshot-location-config region="default" \
--features=EnableCSI
Rest is the same as https://medium.com/techlogs/using-velero-and-restic-to-backup-kubernetes-2f0f812da4db
root@k8s-storage-1:~# k get volumesnapshot --all-namespaces
NAMESPACE NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE
default rbd-pvc-snapshot rbd-pvc csi-rbdplugin-snapclass 6d
test-nginx velero-ceph-ext-bcmqn ceph-ext csi-rbdplugin-snapclass 52m
test3-nginx velero-ceph-ext-lfplp ceph-ext csi-rbdplugin-snapclass 5d23h
root@k8s-storage-1
Status — Not working. Not able to backup
time="2020-09-07T09:58:28Z" level=info msg="Waiting for CSI driver to reconcile volumesnapshot test-nginx/velero-ceph-ext-bcmqn. Retrying in 5s" backup=velero/test-nginx-csi cmd=/plugins/velero-plugin-for-csi logSource="/go/src/velero-plugin-for-csi/internal/util/util.go:172" pluginName=velero-plugin-for-csi
time="2020-09-07T09:58:33Z" level=info msg="Waiting for CSI driver to reconcile volumesnapshot test-nginx/velero-ceph-ext-bcmqn. Retrying in 5s" backup=velero/test-nginx-csi cmd=/plugins/velero-plugin-for-csi logSource="/go/src/velero-plugin-for-csi/internal/util/util.go:172" pluginName=velero-plugin-for-csi
time="2020-09-07T09:58:38Z" level=info msg="Waiting for CSI driver to reconcile volumesnapshot test-nginx/velero-ceph-ext-bcmqn. Retrying in 5s" backup=velero/test-nginx-csi cmd=/plugins/velero-plugin-for-csi logSource="/go/src/velero-plugin-for-csi/internal/util/util.go:172" pluginName=velero-plugin-for-csi
...
time="2020-09-07T10:06:08Z" level=error msg="Timed out awaiting reconciliation of volumesnapshot test-nginx/velero-ceph-ext-bcmqn" backup=velero/test-nginx-csi cmd=/plugins/velero-plugin-for-csi logSource="/go/src/velero-plugin-for-csi/internal/util/util.go:194" pluginName=velero-plugin-for-csi
time="2020-09-07T10:06:08Z" level=info msg="1 errors encountered backup up item" backup=velero/test-nginx-csi logSource="pkg/backup/backup.go:444" name=nginx-test
Bug raised https://github.com/vmware-tanzu/velero-plugin-for-csi/issues/69
root@k8s-storage-1:~# k describe volumesnapshot -n test-nginx
Name: velero-ceph-ext-bcmqn
Namespace: test-nginx
Labels: velero.io/backup-name=test-nginx-csi
Annotations: <none>
API Version: snapshot.storage.k8s.io/v1beta1
Kind: VolumeSnapshot
Metadata:
Creation Timestamp: 2020-09-07T09:56:08Z
Generate Name: velero-ceph-ext-
Generation: 1
Managed Fields:
API Version: snapshot.storage.k8s.io/v1beta1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:generateName:
f:labels:
.:
f:velero.io/backup-name:
f:spec:
.:
f:source:
.:
f:persistentVolumeClaimName:
f:volumeSnapshotClassName:
Manager: velero-plugin-for-csi
Operation: Update
Time: 2020-09-07T09:56:08Z
Resource Version: 3883906
Self Link: /apis/snapshot.storage.k8s.io/v1beta1/namespaces/test-nginx/volumesnapshots/velero-ceph-ext-bcmqn
UID: 64f97657-47cc-45a9-9f3e-fddee6bac425
Spec:
Source:
Persistent Volume Claim Name: ceph-ext
Volume Snapshot Class Name: csi-rbdplugin-snapclass
Events: <none>