Having ReadWriteMany Access Modes in GKE with NFS Server

Warning  FailedAttachVolume  22s   attachdetach-controller  Multi-Attach error for volume "pvc-d8d9f741-9ecb-11ea-a26b-xxxxxxx" Volume is already used by pod(s) nginx-deployment-xxxxx-rdznz, nginx-deployment-xxxxxxx-9wtcm, nginx-deployment-xxxxxxx-4wwqn

We all may have faced this problem while spinning up a workload in Kubernetes with ReadWriteMany(ROX) Access Modes in Persistent Volume Claim. This is a known problem in most of the managed Kubernetes provider platform and most of the vendors have there own preferred solutions to address it. I will be writing this article in accordance with Google Kubernetes Engine (GKE), but this same solution can be applied to the other vendors(Azure Kubernetes Service, Microsoft & Elastic Kubernetes Service by Amazon) if you wish to have solved it with an own managed NFS Server.

In GKE, when you create a PVC, the associated compute disk is created in GCP Persistent Disk with the configuration you mentioned while creating it, like Storage and types. But the claim which you created doesn't support ReadWriteMany AccessModes since the persistent disk doesn't

So this problem can be solved in two ways in NFS (Network File Store) in GKE. Creating a PVC where the backend is Self Managed NFS Server or Cloud FileStore. The cloud filestore seems to be a little expensive and sometimes your architecture might not require it though

First, you may need to create a disk in GCP Persistent Disk with the required storage config, while creating the disk make sure you create the disk on the zone and region where your node for NFS Server will be running and you may need to take into consideration for DRM and regional wide disk which I will write in another article.

or equivalent cloud command

gcloud compute disks create nfs-server-pd-gke --zone=europe-west1-b --size=100GB

optional: attach to a temp instance for organizing folder structures before mounting if required

Create a PersistantVolume in GKE with the disk you created

** I have used default namespace across the example.

Now your persistent volume is ready which is backed back GCP Disk, next is to create a PVC for the volume

Next is to deploy NFS Server and a Service for it

PVC volume will be mounted NFS-server exports directory, the deployment can be customized as per your configs, however, this will be minimum config which you require.

Once this is done, you can create PVC Claims where the backend storage will be NFS Server

This claim can be used in your workloads where you require to have ReadWriteMany access-modes.

Here is your own uncomplicated ReadWriteMany Access Mode PVC. Do let me know if you face any issues. I’m happy to help