Setup EFS with ECS Fargate

In this blog I am going over the process to setup a file system to Fargate. i have wrote previously about setup different services using ECS and Fargate.

With Fargate we can run containers without configure any EC2 instance to run it, all is on the ECS Cluster, this is nice if we only need to run the container without any persistent storage.

But there might be cases when you need to persist some files from your container. For example if you want to keep the logs of your server on a text file.

One option would be to run the container on a EC2, but if we just want to keep with Fargate is possible. We use AWS EFS and attach it to the containers, this EFS volumes can be shared by the different containers at the same time.

Let’s go over the setup of the different parts to run our container on AWS ECS using Fargate and EFS volumes.

Create EFS Volume

First we have to create a security group that allows inbound traffic to NFS.

1. Go to security groups on the EC2 service.
2. Create a new security group
3. Select the VPC where the EFS is going to be used
4. Add Inbound Rules that allow type NFS (port 2049)
5. Add source field.
6. Create the security group and copy the Security Group ID

Now we have to create an EFS Volume

1. Go to EFS service
2. Click on Create file system
3. Select VPC where you have created the security group on step 1
4. On each subnet zone, on the security group section, add the Security Group ID we have created on step 1

EFS Volume network configuration

5. Go next and configure the EFS to you needs, you can leave all the fields as default

6. Create the file system

Configure ECS Task

To setup the task to user the EFS volumes, go to Task Definitions on the ECS section.

1. Create a new task or create a new revision of an existing task
2. On the Volumes section. Add a new volume
3. Type the name you want and select EFS on volume type.
4. Select the on File System ID, the EFS volume we have created previously

Add volume to task definition

1. On the container definitions, click on a container name to edit the configuration.
2. Go to STORAGE AND LOGGING, and on Mount points click on Add mount point
3. Select the volume we added on step 1.3 and add the path to mount the container. (Same a mount a container on docker)

Mount volume on container

1. Update the configuration of the container
2. Click on Create to create a new task revision

Now we have to configure the task

1. You can run the task directly on the cluster or user a service.
2. Configure the task to use Fargate
3. Select version v1.4.0. Latest version of Fargate does not work.
4. Select the VPC and Subnets where you created the security group
5. Run the task.

Container and stored files

Now your containers write on the EFS and the data is persisted across containers. So if you restart or escalate to more containers, all of them will access the same volume. You need to be careful to create the different files, for example if your application is writing log files, and they use the current date to name the file, if different container run at the same time, they will generate the same file name, so your app should handle this cases.

Access the files

In order to access the EFS volume, you have to mount in another system on the VPC.

On the EFS section, click on the file system you have created, and see the details. You can see on _File system access_ some links.

EFS access options

Click in any of them to get the instructions to connect.

One way is to connect through an EC2 instance. Click on Amazon EC2 mount instructions (from local VPC) and follow the instructions to mount the EFS volume. Then you can access the files.

List files from EFS on a EC2 instance

It is easy to attach some volumes to the container, with the downside that you have to access using other services like a EC2, but you can start an EC2 on demand and access the files when needed.