Attach an AWS EFS to an AWS ECS Cluster (from scratch)
So I’m writing this article because I recently ran across a scenario that required this very particular problem of attaching an EFS (Elastic File System) to an auto scaling group of EC2 (Elastic Cloud Compute) all running ECS (Elastic Container Service).
Now as you may know, AWS Fargate manages the execution of your ECS tasks so you don’t need to worry about managing individual EC2’s, however they don’t have the ability to mount an EFS to those tasks (they’re working on it). So for the mean time, if you’d like to have this, you need to do it yourself.
Figuring all this out was quite a hassle to be honest but I learned a lot along the way which is why I’m writing this article and sharing it with you.
Terms & Definitions
These terms are written in my own words, feel free to use the links to learn more about them.
Resources: Tools or services that AWS provides, i.e. EC2, EFS, ECS, etc.
VPC (Virtual Private Cloud): AWS’s way of grouping resources together and providing a layer of security to specific resources so they can’t be accessed outside the VPC.
CIDR (Classless Inter-Domain Routing) Block: A group of IP addresses.
Security Groups: Firewall configurations that allow you to set which IP addresses/ports can access specific resources.
IAM (Identity and Access Management): A place where you can manage users and their access to your AWS account. After creating users, you can assign them a role which is a group of policies that allow certain access to resources. Policies are groups of specific permissions.
AMI (Amazon Machine Images): AMI specifies the type of operating system and other resources your EC2 will run with, such as launch permissions.
NFS (Network File System): A protocol that allows a file system to be shared across multiple computers but be accessed like a local file system.
EC2 Launch Template: This is a blueprint that AWS will use to create an EC2 for you. You can define the AMI,
EC2 Auto Scaling Group: A collection of EC2s that can scale up or down according to the groups policy.
User data scripts: A script that contains code you want to run at the launch of each EC2. This can be defined in such a way where it only runs once per AMI, once per instance or on every boot cycle.
What you came here for
Things I cover in this article:
- Create a VPC, Subnets, Route Table & Internet Gateway
- Create Security Groups
- Create IAM Roles (optional)
- Create an EFS
- Create an empty ECS Cluster
- Create a EC2 Launch Template with user data scripts
Things I won’t cover in this article:
- How to tear it all down so you don’t get charged 😬
Note: All resources should be created under the same AWS region. The rest of this article will be done within the AWS Console.
Create a VPC, Subnets, Route Table & Internet Gateway
Let’s start by creating the VPC. As a matter of fact, all resources used in this article must be within the same VPC otherwise it may not work.
Start by going to the VPC Dashboard and click “Create VPC”. Give the VPC a name, something descriptive. I like to prefix all AWS resources with the project or company their associate with. Then assign the IPv4 CIDR Block to 10.0.0.0/16. Leave the rest at their defaults and click create.
Select the newly created VPC and click Actions > Edit DNS hostnames. Under the Edit DNS hostnames page, check enable DNS hostnames and click save. This will create DNS hostnames for your EC2 resources so you can access them via SSH if needed.
Next create an internet gateway by going to the Internet Gateway tab under the VPC dashboard and clicking “Create internet gateway”. Give the gateway a name and click create.
Select the newly created Internet Gateway and under Actions, select “Attach to VPC”. In the next page select the newly created VPC and click Attach.
Go to the Route Table tab in the VPC dashboard, select the newly created route table (that was created with the VPC) and under Actions, select “Edit routes”.
Add a new route to the Route table which routes all traffic (0.0.0.0/0) to the newly created Internet Gateway. Click save routes.
In this next step we need to create 2 subnets. The first subnet will be taking care of the 10.0.0.0/20 block and the second will be handling the 10.0.16.0/20 block. This essentially splits the VPC CIDR block into two subnets. These should be put on two different Availability zones if possible.
With the two newly created Subnets, you want to modify the auto-assign IP settings by selecting each subnet and under actions selecting “Modify auto-assign IP settings”. Then from the edit page check the Auto-assign IPv4 to enable and click save. Doing this will auto-assign an IPv4 public IP to resources such as your EC2 instances so they can be accessed from the outside world for reasons such as web servers or just SSH.
Creating Security Groups
We need to create two security groups. The first will be for the ECS cluster and its EC2 instances. The second will be for the EFS.
Under the VPC dashboard, navigate to the Security Group tab and click “Create security group”. First create the ECS security group. Give it a name and description and attach it to the VPC we created earlier. Then repeat the same steps for the EFS security group.
After creating the two security groups, we need to edit the inbound rules for the EFS security group to allow NFS traffic from the ECS security group. What this does is allows to mount and access the EFS from your ECS EC2 instances.
The way we do this is by copying the Security Group ID of the ECS security group we created and then place that in the source field of the inbound rule for the EFS security group. The type here is NFS which specifies TCP traffic on port 2049.
Creating IAM Roles (optional)
TODO…
Create an EFS
In this step we’ll create a new EFS.
Navigate to the EFS dashboard and click “Create file system”. For VPC, select the new VPC we created. For the subnets, we use the new subnets we created with the VPC. And for the security group, we’ll attach the newly create EFS security group. The rest of the configuration options can be left to default and we’ll create the EFS.
Create an empty ECS Cluster
In this step we will create an empty ECS cluster. The reason we leave it empty is otherwise AWS will spin up an entire Cloud Formation stack including a VPC, auto-scaling group of EC2 instances, etc. but leaving out key steps like attaching a template to the EC2 instances which is how we auto attach the EFS to our EC2s.
Under the ECS dashboard navigate to the Clusters tab and click “Create Cluster”. You’ll have the choice of selecting a “cluster template” and for this choose “EC2 Linux + Networking” as the template. Next under the configure cluster step give your cluster a name, check “Create an empty cluster” and click “Create”.
Where are we at right now and what’s next?
So far, we’ve created a new VPC, Route Table, Subnets & Internet Gateway so that all our resources can be behind one VPC and all traffic can be routed appropriately.
We’ve created Security Groups to handle our traffic between resources and from the outside world.
We created some IAM Roles with specific policies to explicitly allow resources access to read/write other AWS resources.
We created an EFS to store things on a shared file system that can be attached to virtually and unlimited number of EC2s.
We created an empty ECS cluster that we can now reference and assign tasks to.
However, before we can assign tasks and services to the empty ECS cluster, we need to setup our auto-scaling EC2 group and run the ECS service on it, as well as auto mounting the EFS on boot.
Create a EC2 Launch Template with user data scripts
Creating the EC2 Launch Template is pretty straightforward. The important step in this however, is adding the user data script to the template to run some scripts on boot.
Navigate to the EC2 dashboard and go to the Launch Template tab. From there, click “Create Launch Template”. In here, we’ll create a new launch template. Give the template a name and description. Do not specify a source template. For the AMI ID you’ll need to find an AMI ID from the AMI Marketplace or EC2 Instance create page. In this example I’m using the Amazon Linux 2 AMI (HVM), SSD Volume Type. For instance type choose t3a.micro or whichever you prefer. If you’d like to SSH into the server, select a Key pair name or create one. For Network type select VPC and for Security Group, choose the ECS Security Group we created earlier.
Next scroll to the Advanced Details section of creating a launch template and under the IAM Instance Profile, select the ECS Role created earlier. If you didn’t create one, you can use any role that has the AmazonEC2ContainerServiceforEC2Role policy attached to it, such as the ecsInstanceRole created by AWS.
Finally we add the user data script. This is important as it will automatically mount the EFS to each EC2 that we run. Additionally it will start the ECS service and attach it to our cluster.
Content-Type: multipart/mixed; boundary="==BOUNDARY=="
MIME-Version: 1.0--==BOUNDARY==
Content-Type: text/cloud-boothook; charset="us-ascii"# Mount /mnt/efs
cloud-init-per instance add_efs_to_fstab echo -e 'fs-XXXXXXXX:/ /mnt/efs efs defaults,_netdev 0 0' >> /etc/fstab# Mkdir /etc/ecs
cloud-init-per instance mkdir_ecs mkdir /etc/ecs# write ecs.config
cloud-init-per instance write_ecs_config echo -e "ECS_CLUSTER=Test-Cluster" >> /etc/ecs/ecs.config--==BOUNDARY==
Content-Type: text/x-shellscript; charset="us-ascii"#!/bin/bash
yum update -y
yum install -y amazon-efs-utils git
amazon-linux-extras install -y ecs
mkdir -p /mnt/efsif [[ ! $(mount | grep fs-XXXXXXXX) ]]; then
mount -a
fi# Start ecs service
systemctl enable --now ecs &--==BOUNDARY==--
TODO: Break down what this script is doing.
Summary
So that’s basically it! You’re now setup to run a service on an AWS Cluster using ECS and automatically have an EFS mounted on each EC2 they’re running on.
Because the EFS is mounted to the EC2, each container running with ECS on that EC2 will have access to mount the EFS and use its contents. I’ve been successfully using this model to run a group of EC2s with containers running on each and they all share the same EFS.
Thanks so much for taking the time to read this. Leave a comment for any questions or notes.
