Managing Persistent Storage with Amazon S3 on Amazon EKS

In this article, we will explore how to manage persistent storage on Amazon EKS using the Mountpoint for Amazon S3 CSI Driver. This driver allows Kubernetes pods to mount S3 buckets as file systems, enabling applications to access S3 objects seamlessly. This integration can significantly enhance the flexibility and scalability of your applications by leveraging S3’s durability and availability.

Prerequisites

Before we begin, ensure you have the following:

• AWS account with necessary permissions
• EKS cluster
• kubectl configured to interact with your EKS cluster
• AWS CLI installed and configured
• Helm installed on your machine

Step 1: Install the Mountpoint for S3 CSI Driver using Helm

First, we need to install the CSI driver on our EKS cluster using Helm. Use the following commands to deploy the necessary resources:

1. Add Helm chart repository:

helm repo add aws-mountpoint-s3-csi-driver https://awslabs.github.io/mountpoint-s3-csi-driver
helm repo update

2. Install the latest release of the driver:

helm upgrade --install aws-mountpoint-s3-csi-driver \
    --namespace kube-system \
    aws-mountpoint-s3-csi-driver/aws-mountpoint-s3-csi-driver

3. Once the driver has been deployed, verify the pods are running:

kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-mountpoint-s3-csi-driver

Step 2: Create an S3 Bucket

We need to create an S3 bucket to store our data.

aws s3api create-bucket --bucket your-bucket-name --region your-region

Replace your-bucket-name with a unique name for your bucket and your-region with the AWS region you want to use.

Step 3: Create an IAM Role for the Driver

The CSI driver requires permissions to interact with S3. Follow these steps to create an IAM policy, role, and associate it with the service account used by the CSI driver.

1. Create an IAM policy JSON file s3-csi-policy.json with the necessary permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        }
    ]
}

2. Create the IAM policy using AWS CLI:

aws iam create-policy --policy-name AmazonS3CSIPolicy --policy-document file://s3-csi-policy.json

3. Create an IAM role with a trust policy allowing EKS to assume this role. Save the trust policy to a file trust-policy.json:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "eks.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

4. Create the role using AWS CLI:

aws iam create-role --role-name AmazonS3CSIRole --assume-role-policy-document file://trust-policy.json

5. Attach the previously created policy to the role:

aws iam attach-role-policy --role-name AmazonS3CSIRole --policy-arn arn:aws:iam::aws-account-id:policy/AmazonS3CSIPolicy

6. Associate this role with the service account used by the CSI driver:

eksctl create iamserviceaccount \
  --name s3-csi-controller-sa \
  --namespace kube-system \
  --cluster your-cluster-name \
  --attach-role-arn arn:aws:iam::aws-account-id:role/AmazonS3CSIRole \
  --approve \
  --override-existing-serviceaccounts

Step 4: Configure the StorageClass

Create a StorageClass to define how the storage should be provisioned:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: s3-csi
provisioner: s3.csi.aws.com
parameters:
  type: standard

Apply this configuration:

kubectl apply -f storageclass.yaml

The StorageClass defines the provisioning parameters for the S3-backed volumes. It specifies the type of S3 storage to be used and other relevant settings.

Step 5: Create a PersistentVolume and PersistentVolumeClaim

Next, create a PersistentVolume (PV) and a PersistentVolumeClaim (PVC) to use the storage class:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-s3
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: s3-csi
  csi:
    driver: s3.csi.aws.com
    volumeHandle: your-bucket-name
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-s3
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: s3-csi
  resources:
    requests:
      storage: 5Gi

Apply the configurations:

kubectl apply -f pv.yaml
kubectl apply -f pvc.yaml

The PersistentVolume specifies the storage details and the backing S3 bucket, while the PersistentVolumeClaim allows pods to request the storage defined by the PV.

Step 6: Deploy a Sample Application

Finally, deploy a sample application to use the PVC:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: s3-storage
      volumes:
      - name: s3-storage
        persistentVolumeClaim:
          claimName: pvc-s3

Apply the deployment:

kubectl apply -f deployment.yaml

This deployment creates an Nginx pod that mounts the S3-backed PVC at /usr/share/nginx/html. This setup demonstrates how applications can use S3 storage seamlessly within EKS.

Conclusion

You have successfully configured the Mountpoint for Amazon S3 CSI Driver on your EKS cluster and deployed a sample application using S3-backed storage. This setup allows your applications to seamlessly interact with S3 buckets as file systems, enhancing your data management capabilities on AWS.