Empowering Infrastructure: An In-Depth Look at EFS, Automation and Best Practices

Introduction

This blog provides solution for lightning-fast data retrieval and writing, coupled with stringent low-latency requirements using EFS . This guide delves into the intricacies of managing permissions within the filesystem, addressing scenarios where LDAP isn’t in play. This will walk you through the process of seamlessly working with Amazon Elastic File System (EFS) and provide a detailed automation guide for effortlessly mounting EFS across any number of instances within your infrastructure. Additionally, gain insights into approach for backup and data restoration within EFS using AWS’s dedicated backup service.

EFS ( Elastic file systems)

Elastic File System (EFS) is a scalable, fully managed file storage service provided by Amazon Web Services (AWS). It’s designed to provide highly available and durable file storage that can be accessed concurrently by multiple Amazon Elastic Compute Cloud (EC2) instances and on-premises servers.
How EFS works — https://docs.aws.amazon.com/efs/latest/ug/how-it-works.html

Setup EFS and Mount EFS by Ansible playbook on servers

• Create elastic file system storage from the AWS console. Ensure we use the same VPC where the EFS has to be mounted on the node. Keep default standard storage class only.
• For mounting EFS the amazon-efs-utils tool should be installed on the nodes. The amazon-efs-utils package is an open-source collection of Amazon EFS tools that is also referred to as the Amazon EFS client.
  To build and install amazon-efs-utils as a Debian package for Ubuntu and Debian we can refer doc Manually installing the Amazon EFS client — Amazon Elastic File System
• Below is the automation done for mounting efs on the different servers via ansible playbook. Run the playbook it will mount efs.

vars:
    efs_utils_download_source_url: "https://github.com/aws/efs-utils"
    efs_utils_local_path: "/tmp/efs-utils"
    efs_mount_dir: "/efs"
    efs_utils_manage_config: false
    nfs_package: "amazon-efs-utils"
    efs_file_system_id: "<efs_id>"

  tasks:
    - name: Install Debian build dependencies
      package:
        name: binutils
        state: present
        update_cache: yes
      become: yes

    - name: Clone efs-utils source repo
      git:
        repo: "{{ efs_utils_download_source_url }}"
        dest: "{{ efs_utils_local_path }}"
        clone: yes
        update: yes

    - name: build deb package
      shell: "{{ efs_utils_local_path }}/build-deb.sh"
      args:
        chdir: "{{ efs_utils_local_path }}"
        creates: build/amazon-efs-utils*deb

    - name: install deb package
      shell: apt -y install ./build/amazon-efs-utils*deb
      args:
        chdir: "{{ efs_utils_local_path }}"
      become: yes

    - name: ensure NFS is installed.
      package:
        name: "{{ nfs_package }}"
        state: installed
    
    - name: create mount directory
      file:
        path: "{{ efs_mount_dir }}"
        state: directory
        mode: 0755
        owner: root
        group: root

    - name: mount EFS
      mount:
        fstype: efs
        opts: _netdev,tls
        src: "{{ efs_file_system_id }}:/"
        path: "{{ efs_mount_dir }}"
        state: mounted

• Check whether the EFS is mounted or not. It should be mounted with 127.0.0.1:/ filesystem. Check by df -hcommand on the server.
• After creating a file system, by default only the root user (UID 0) has read, write, and execute permissions. For other users to modify the file system, explicitly create the folder inside the /efsand give the permission of the required user.

Permissions with non LDAP user-management system

In environments where LDAP is not present in the self managed infra, it’s common for the same user to have different User IDs (UIDs) on different nodes. This discrepancy can lead to permission issues when mounting EFS and impact the seamless reading and writing of data across various nodes.

To preemptively tackle this hurdle, it’s essential to synchronize user UIDs across all nodes before mounting EFS. This proactive measure ensures smooth data access and avoids potential permission conflicts, setting the stage for optimized EFS utilization within your infrastructure.

EFS Backup and Recovery from backup

We can use the AWS backup service to backup and restore the EFS data.

AWS Backup is a fully-managed backup service that makes it easy to centralize and automate data backup across AWS services. It enables backup and restores data across AWS services, on-premises software, and hybrid environments.

With AWS Backup, we can create backup plans that define the backup schedule, retention period, and backup window for your resources. We can choose from various backup options such as full backups, incremental backups, and differential backups. AWS Backup also offers features like versioning, cross-region replication, and lifecycle management to manage backups efficiently.

The restore feature in AWS Backup allows recovering data from backups in the event of data loss or corruption. We can restore the entire backup or just specific files or folders.

To read more about it: What is AWS Backup? — AWS Backup

Create Backup Plan

1. Choose AWS backup service and Create a backup plan.
   Create a new vault for efs and set the backup frequncy and retention period.

2. After making the above changes, in the section of the backup vault and check the access policy. By default, it denies all the actions mentioned in the policy. Edit it explicitly and Allow all the actions.

Restore EFS file system

1. Go to Protected resources under AWS backup service. Click on the file system, we can see multiple backups happening for the file system. Select any of the backups and click on restore. There are 2 ways we can restore, according to the issues and requirements we can choose the restore location. Choose default role only and select Restore Backup

The status of the restore can be seen in jobs section under AWS backup service

Once it is completed we can check in the Elastic file system. A new file system will be created.

2. Click on the newly created file system and check the network section. It should be the same as the filesystem from which the EFS is backed up. The network configuration should be the same for both file systems.
If the network configuration is not the same we can edit it by clicking on manage and make it the same as the actual file system

3. Mount this newly created file system on the node again. Click on attach.