Restore InfluxDB from backup in kubernetes

Restoring InfluxDB v1.3 from backup in Docker environment is no small feat (duh). But Kubernetes isn’t exactly docker — so how do we do it there?

The idea in docker variant is simple: have 2 volumes, one for InfluxDB data (without it you wouldn’t have persistence anyway) and one for backup files. Place files into backup volume, stop your main container, launch its copy, but do not launch InfluxDB service, only run restore command. Then start main container and done.

We will do practically the same.

1. Create PersistentVolumeClaim for backup files (tune size if necessary):

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: influxdb-backup
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

2. Modify your InfluxDB deployment:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: influxdb
  labels:
    name: influxdb
...
      volumes:
      - name: influx
        persistentVolumeClaim:
          claimName: influxdb
      - name: influx-backup
        persistentVolumeClaim:
          claimName: influxdb-backup      
      containers:
      - name: influxdb
        image: "influxdb:alpine"
        volumeMounts:
        - mountPath: /var/lib/influxdb
          name: influx
        - mountPath: /tmp/backup
          name: influx-backup
...

3. Now apply the changes via kubectl and copy your backup files via kubectl cp:

kubectl cp <local_path> <namespace>/<pod_name>:/tmp/backup/

4. Delete your InfluxDB deployment (ouch), because InfluxDB needs to be stopped for import

5. Create job which uses same container image and volume setup, but modify its command:

kind: Job
metadata:
  name: influx-restore
spec:
  template:
    metadata:
      name: influx-restore
      labels:
        task: influx-restore
    spec:
      volumes:
      - name: influx
        persistentVolumeClaim:
          claimName: influxdb
      - name: backup
        persistentVolumeClaim:
          claimName: influx-backup
      containers:
        - name: influx
          image: "influxdb:alpine"
          command: ["/bin/sh"]
          args: ["-c", "influxd restore -metadir /var/lib/influxdb/meta -database <your_db_here> -datadir /var/lib/influxdb/data /tmp/backup/"]
          volumeMounts:
          - mountPath: /var/lib/influxdb
            name: influx
          - mountPath: /tmp/backup
            name: backup
      restartPolicy: Never

6. Apply job config and check if it ran successfully (check pod’s log)

7. If everything was ok, you can recreate your InfluxDB deployment and check if it worked via CLI or HTTP.

8. Remove backup volume persistent claim and remove its use from deployment config.

AFAIK, using initContainer should work too, but I was more familiar with Jobs so that was my choice.