Solutions for Operation not permitted error when Postgresql is running on Docker using Vagrant’s NFS type vm.synced_folder
Problem description
When PostgreSQL is runnign on Docker and using NFS volume for its persistent storage, Operation not permitted error occurs when PostgreSQL container try to change owner of directory at its boot time.
chown: changing ownership of '/var/lib/postgresql/data': Operation not permitted
Above error leads another error :
could not translate host name "db" to address: Name or service not known
Root cause
This is because there is a mismatch between NFS settings , uid/gid inside PostgreSQL container and owner of nfs shared db directory.
Since root_squash
is a default settings of NFS, root
on PostgreSQL container treat as uid=1000
, gid=1000
and this caused anOperation not permitted
error.
If you aren’t using Vagrant, root
on PostgreSQL container mapped to 65534:65534
, a.k.a nobody:nogroup
or maybe nfsnobody:nfsnogroup
on some distribution. You can modify thismapping
behaviour of NFS by using anonuid
and anonguid
settings, and Vagrant setsanonuid=1000
and anonguid=1000
to NFS settings automatically behind the scene, so that squashed root
treat as 1000:1000
above case.
Example NFS settings that Vagrant produced is shown below:
# VAGRANT-BEGIN: 1000 20a43a35-1bf9-4c57-afe9-3ddbed483aa1
"/home/ubuntu/kvm/django3/shared" 192.168.121.59(rw,anonuid=1000,anongid=1000,fsid=4255092591)
# VAGRANT-END: 1000 20a43a35-1bf9-4c57-afe9-3ddbed483aa1
And that leads this uid/gid mismatch situation:
[vagrant@django3 shared]$ docker-compose run db ls -la /var/lib/postgresql
total 4
drwxr-xr-x. 1 postgres postgres 18 Mar 29 04:35 .
drwxr-xr-x. 1 root root 24 Mar 29 04:34 ..
drwx------. 2 1000 1000 4096 Apr 20 02:25 data
To PostgreSQL container succesfully boot up, There are 2 solutions at least.
Solution 1: use no_root_squash option (not recommended)
If no_root_squash
is set to NFS settings, root
on PostgreSQL contanier treat as uid=0
, gid=0
so that chown
and chmod
command won’t fail and PostgreSQL container boot up as usual. But this method isn’t recommended on security point of view. Example settings on Vagrantfile is shown below:
config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.synced_folder "shared", "/srv/shared", type: "nfs",
mount_options: ['rw', 'vers=3', 'tcp'],
linux__nfs_options: ['rw','no_root_squash']
Solution 2: live with root_squash
To live with root_squash
and 'anonuid=1000','anongid=1000'
, what can we do when we need to give a right permissions to postgres (999:999)
user on its container at the same time?
Let’s NFS settings on Vagrantfile to be minimal:
config.vm.synced_folder "shared", "/srv/shared", type: "nfs"
or you can write default settings explicitly:
config.vm.synced_folder "shared", "/srv/shared", type: "nfs",
mount_options: ['rw', 'vers=3', 'tcp'],
linux__nfs_options: ['rw','root_squash','anonuid=1000','anongid=1000']
Change owner of data
directory to 999:999
on NFS server ( This time Vagrant host) system. data
directory is going to use for persistent volume for PostgreSQL. 999:999
is matched to postgre
user on PosgreSQL container.
sudo chown -R 999:999 data
Example docker-compose.yml
, that default user changed to 999:999
from root
, is shown below:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
user: "999:999"
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
Let’s boot up PostgreSQL container with docker-compose up db
, PostgreSQL container will boot up successfully :)
Reference
I used above Django example for this test.
Here I attached my example Vagrantfile and docker-compose.yml.