Solutions for Operation not permitted error when Postgresql is running on Docker using Vagrant’s NFS type vm.synced_folder

Problem description

When PostgreSQL is runnign on Docker and using NFS volume for its persistent storage, Operation not permitted error occurs when PostgreSQL container try to change owner of directory at its boot time.

chown: changing ownership of '/var/lib/postgresql/data': Operation not permitted

Above error leads another error :

could not translate host name "db" to address: Name or service not known

Root cause

This is because there is a mismatch between NFS settings , uid/gid inside PostgreSQL container and owner of nfs shared db directory.

Since root_squash is a default settings of NFS, rooton PostgreSQL container treat as uid=1000 , gid=1000 and this caused anOperation not permitted error.

If you aren’t using Vagrant, rooton PostgreSQL container mapped to 65534:65534 , a.k.a nobody:nogroup or maybe nfsnobody:nfsnogroupon some distribution. You can modify thismapping behaviour of NFS by using anonuid and anonguid settings, and Vagrant setsanonuid=1000 and anonguid=1000 to NFS settings automatically behind the scene, so that squashed root treat as 1000:1000 above case.

Example NFS settings that Vagrant produced is shown below:

# VAGRANT-BEGIN: 1000 20a43a35-1bf9-4c57-afe9-3ddbed483aa1
"/home/ubuntu/kvm/django3/shared" 192.168.121.59(rw,anonuid=1000,anongid=1000,fsid=4255092591)
# VAGRANT-END: 1000 20a43a35-1bf9-4c57-afe9-3ddbed483aa1

And that leads this uid/gid mismatch situation:

[vagrant@django3 shared]$ docker-compose run db ls -la /var/lib/postgresql
total 4
drwxr-xr-x. 1 postgres postgres   18 Mar 29 04:35 .
drwxr-xr-x. 1 root     root       24 Mar 29 04:34 ..
drwx------. 2     1000     1000 4096 Apr 20 02:25 data

To PostgreSQL container succesfully boot up, There are 2 solutions at least.

Solution 1: use no_root_squash option (not recommended)

If no_root_squash is set to NFS settings, rooton PostgreSQL contanier treat as uid=0 , gid=0 so that chown and chmod command won’t fail and PostgreSQL container boot up as usual. But this method isn’t recommended on security point of view. Example settings on Vagrantfile is shown below:

config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.synced_folder "shared", "/srv/shared", type: "nfs",
    mount_options: ['rw', 'vers=3', 'tcp'],
    linux__nfs_options: ['rw','no_root_squash']

Solution 2: live with root_squash

To live with root_squash and 'anonuid=1000','anongid=1000' , what can we do when we need to give a right permissions to postgres (999:999) user on its container at the same time?

Let’s NFS settings on Vagrantfile to be minimal:

config.vm.synced_folder "shared", "/srv/shared", type: "nfs"

or you can write default settings explicitly:

config.vm.synced_folder "shared", "/srv/shared", type: "nfs",
  mount_options: ['rw', 'vers=3', 'tcp'],
  linux__nfs_options: ['rw','root_squash','anonuid=1000','anongid=1000']

Change owner of data directory to 999:999 on NFS server ( This time Vagrant host) system. data directory is going to use for persistent volume for PostgreSQL. 999:999 is matched to postgre user on PosgreSQL container.

sudo chown -R 999:999 data

Example docker-compose.yml , that default user changed to 999:999 from root, is shown below:

db:
  image: postgres
  volumes:
   - ./data/db:/var/lib/postgresql/data
  user: "999:999"
  environment:
    - POSTGRES_DB=postgres
    - POSTGRES_USER=postgres
    - POSTGRES_PASSWORD=postgres

Let’s boot up PostgreSQL container with docker-compose up db, PostgreSQL container will boot up successfully :)

Reference

Quickstart: Compose and Django

I used above Django example for this test.
Here I attached my example Vagrantfile and docker-compose.yml.