Migrate Docker volume from one Mac OSX host to another

Posted also in my personal blog.

Recently I switched my MacBook Pro laptop with a newer one, Yay! I had regular time machine backups on an external drive, thus I thought the transition will be super smooth. Spoiler: it wasn’t.

I switched over to my project’s directory, ran docker-compose up and expected the magic to happen. My project started just fine except for the fact that the named volume I had in my old machine wasn’t migrated to the new machine. Now what?

I listed the volumes I have on my old machine by running docker volume ls, located the relevant volume, let’s call it data-vol and then ran docker volume inspect data-vol. The above command printed something like this:

[
    {
        "CreatedAt": "2018-10-10T09:53:30Z",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "project",
            "com.docker.compose.version": "1.22.0",
            "com.docker.compose.volume": "data-vol"
        },
        "Mountpoint": "/var/lib/docker/volumes/data-vol/_data",
        "Name": "data-vol",
        "Options": null,
        "Scope": "local"
    }
]

Great! All I need to do is copy /var/lib/docker/volumes/data-vol to the same path on the new machine, and everything will work. How simple is that?

Not quite. Apparently, /var/lib/docker is hidden under the xhyve VM (the virtual machine manager that is used by docker-machine on Mac OSX). I needed to implement some workarounds to reach it.

Solution

Exporting the volumes on the old machine

I found an easy way to accomplish what I want in this thread:

docker run --rm -it -v ~/volume-backup:/backup -v /var/lib/docker:/docker alpine:edge tar cfz /backup/volumes.tgz /docker/volumes/

The above command runs an alpine container with two volumes:

1. (host) ~/volume-backup => (container) /backup
2. (host) /var/lib/docker/ => (container) /docker

This container executes a single command which archives /docker/volumes and I will eventually end up with a volumes.tgz file in my ~/volume-backup folder (on my host).

Note: This might take a while in case you have many docker volumes or large volumes.

Importing the volumes in the new machine

After copying volumes.tgz file to my new machine (same path, ~/volume-backup/volumes.tgz), I needed (again) to access /var/lib/docker in order to deploy this backup file.

I navigated to the path where the archive is:

cd ~/volumes-backup

And extracted the archive by running:

tar zxvf volumes.tgz

At this point, all of the volumes will be available under ./docker/volumes.

Note: This might take a while in case you have many docker volumes or large volumes.

Important Note: Before moving on, make sure there’s no running container which uses the volume data-vol (docker stop such containers).

Similarly to what I did when exporting the volumes in the previous section, I now ran:

docker run --rm -it -v /var/lib/docker:/docker -v ~/volume-backup/docker/volumes:/volume-backup alpine:edge cp -r /volume-backup/data-vol /docker/volumes

The above command runs an alpine container with two volumes:

1. (host) /var/lib/docker => (container) /docker
2. (host) ~/volume-backup => (container) /volume-backup/docker/volumes

This container executes a single command which copies the volume file to its correct location in the new machine.

At this point, running docker-compose up started my project with all the data residing where it belongs.

Conclusion

When working with docker-machine on a Mac OSX, /var/lib/docker is not accessible directly from the host machine. In most cases, you don’t need to access this path.

However, if you do need to access it, the workaround is running a light container with two volumes, one that its source is /var/lib/docker on the host machine, which is accessible to docker’s VM and the second one’s source is the path where you want to take files from/to.