Published in


Having ReadWriteMany Access Modes in GKE with NFS Server

Warning  FailedAttachVolume  22s   attachdetach-controller  Multi-Attach error for volume "pvc-d8d9f741-9ecb-11ea-a26b-xxxxxxx" Volume is already used by pod(s) nginx-deployment-xxxxx-rdznz, nginx-deployment-xxxxxxx-9wtcm, nginx-deployment-xxxxxxx-4wwqn

We all may have faced this problem while spinning up a workload in Kubernetes with ReadWriteMany(ROX) Access Modes in Persistent Volume Claim. This is a known problem in most of the managed Kubernetes provider platform and most of the vendors have there own preferred solutions to address it. I will be writing this article in accordance with Google Kubernetes Engine (GKE), but this same solution can be applied to the other vendors(Azure Kubernetes Service, Microsoft & Elastic Kubernetes Service by Amazon) if you wish to have solved it with an own managed NFS Server.

In GKE, when you create a PVC, the associated compute disk is created in GCP Persistent Disk with the configuration you mentioned while creating it, like Storage and types. But the claim which you created doesn't support ReadWriteMany AccessModes since the persistent disk doesn't

So this problem can be solved in two ways in NFS (Network File Store) in GKE. Creating a PVC where the backend is Self Managed NFS Server or Cloud FileStore. The cloud filestore seems to be a little expensive and sometimes your architecture might not require it though

First, you may need to create a disk in GCP Persistent Disk with the required storage config, while creating the disk make sure you create the disk on the zone and region where your node for NFS Server will be running and you may need to take into consideration for DRM and regional wide disk which I will write in another article.

or equivalent cloud command

gcloud compute disks create nfs-server-pd-gke --zone=europe-west1-b --size=100GB

optional: attach to a temp instance for organizing folder structures before mounting if required

Create a PersistantVolume in GKE with the disk you created

** I have used default namespace across the example.

Now your persistent volume is ready which is backed back GCP Disk, next is to create a PVC for the volume

Next is to deploy NFS Server and a Service for it

PVC volume will be mounted NFS-server exports directory, the deployment can be customized as per your configs, however, this will be minimum config which you require.

Once this is done, you can create PVC Claims where the backend storage will be NFS Server

This claim can be used in your workloads where you require to have ReadWriteMany access-modes.

Here is your own uncomplicated ReadWriteMany Access Mode PVC. Do let me know if you face any issues. I’m happy to help

Official Adfolks LLC Publications.

Recommended from Medium

Introduction to Kubernetes: Part 2

2.5D Platformer

Object Oriented Programming- concepts and problem solving.

How to setup Ansible Dynamic Inventory for Docker using Ansible Plugin?

Final Report

AZ-900 Microsoft Azure Fundamentals. My experience.

Introducing Azure Load Balancer Insights using Azure Monitor for Networks

Exporting Substance Painter GLTFs to A-Frame

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Tibin Lukose

Tibin Lukose

Cloud Native Engineering & Security Enthusiast

More from Medium

Announcing Aleo Setup

Nginx ingress controller reporting error “Failed to update lock”

Working with Objects in AirData Tables

Embedded System : Input / Output with ESP32