In this blog I am going over the process to setup a file system to Fargate. i have wrote previously about setup different services using ECS and Fargate.
With Fargate we can run containers without configure any EC2 instance to run it, all is on the ECS Cluster, this is nice if we only need to run the container without any persistent storage.
But there might be cases when you need to persist some files from your container. For example if you want to keep the logs of your server on a text file.
One option would be to run the container on a EC2, but if we just want to keep with Fargate is possible. We use AWS EFS and attach it to the containers, this EFS volumes can be shared by the different containers at the same time.
Let’s go over the setup of the different parts to run our container on AWS ECS using Fargate and EFS volumes.
Create EFS Volume
First we have to create a security group that allows inbound traffic to NFS.
- Go to security groups on the EC2 service.
- Create a new security group
- Select the VPC where the EFS is going to be used
- Add Inbound Rules that allow type NFS (port 2049)
- Add source field.
- Create the security group and copy the Security Group ID
Now we have to create an EFS Volume
- Go to EFS service
- Click on Create file system
- Select VPC where you have created the security group on step 1
- On each subnet zone, on the security group section, add the Security Group ID we have created on step 1
5. Go next and configure the EFS to you needs, you can leave all the fields as default
6. Create the file system
Configure ECS Task
To setup the task to user the EFS volumes, go to Task Definitions on the ECS section.
- Create a new task or create a new revision of an existing task
- On the Volumes section. Add a new volume
- Type the name you want and select EFS on volume type.
- Select the on File System ID, the EFS volume we have created previously
- On the container definitions, click on a container name to edit the configuration.
- Go to STORAGE AND LOGGING, and on Mount points click on Add mount point
- Select the volume we added on step 1.3 and add the path to mount the container. (Same a mount a container on docker)
- Update the configuration of the container
- Click on Create to create a new task revision
Now we have to configure the task
- You can run the task directly on the cluster or user a service.
- Configure the task to use Fargate
- Select version v1.4.0. Latest version of Fargate does not work.
- Select the VPC and Subnets where you created the security group
- Run the task.
Container and stored files
Now your containers write on the EFS and the data is persisted across containers. So if you restart or escalate to more containers, all of them will access the same volume. You need to be careful to create the different files, for example if your application is writing log files, and they use the current date to name the file, if different container run at the same time, they will generate the same file name, so your app should handle this cases.
Access the files
In order to access the EFS volume, you have to mount in another system on the VPC.
On the EFS section, click on the file system you have created, and see the details. You can see on _File system access_ some links.
Click in any of them to get the instructions to connect.
One way is to connect through an EC2 instance. Click on Amazon EC2 mount instructions (from local VPC) and follow the instructions to mount the EFS volume. Then you can access the files.
It is easy to attach some volumes to the container, with the downside that you have to access using other services like a EC2, but you can start an EC2 on demand and access the files when needed.