EFS on ECS Fargate

3 min readMay 26, 2020

Or when LATEST is not the latest. A Terraform story

Background

In April 2020, AWS announced:

Amazon Elastic Container Service (ECS) tasks running on both Amazon Elastic Compute Cloud (EC2) and AWS Fargate can now mount Amazon Elastic File System (EFS) file systems.

This is big news for users of ECS Fargate as persistent storage so far was very limited:

10 GB of Docker layer storage
An additional 4 GB for volume mounts

while ECS tasks can disappear (conceptually) anytime. This makes it ECS-Fargate not a great fit for low performance database and anything else that requires durable storage for longer periods of time while ensuring easy accessibility.

With EFS now being available, bigger storage should be available for reasonable price: EFS charges on GB-month basis (unlike the blocke storage in EBS).

Also access to volumes in in ECS is (like Docker) limited to other ECS tasks only, whereas EFS can be accessed by EC2, too.

Usage

With all this being said, let’s use a simple example that showcases the usage of EFS in ECS Fargate with a InfluxDB task to achieve this:

We use Terraform as Infrastructure-as-Code tool:

Terraform Example

First we need an EFS file system (think about it like a NFS share). The Name tag is used to put a proper name on the resource in the AWS console:

locals {
  # Common tags to be assigned to all resources for this exercise
  common_tags = {
    Project = "Medium"
    Env     = terraform.workspace
  }
}resource "aws_efs_file_system" "influxdb" {
  tags = merge(local.common_tags, {
    Name = "Example-${terraform.workspace}-influxdb"
  })
}

The EFS must also be made available on the same subnet that will be used for the ECS task:

resource "aws_efs_mount_target" "influxdb" {
  count = length(aws_subnet.public)
  file_system_id  = aws_efs_file_system.influxdb.id
  subnet_id       = aws_subnet._[count.index].id
  security_groups = [aws_security_group.efs_influxdb_access.id]
}

and allow access to it only for the one security group:

resource "aws_security_group" "efs_influxdb_access" {
  name        = "Example-${terraform.workspace}-EFS-influxdb-access"
  description = "Allow access to the Influxdb EFS"
  vpc_id      = aws_vpc._.id  ingress {
    from_port = 2049
    to_port   = 2049
    protocol  = "tcp"
    security_groups = [ aws_security_group.influxdb_access.id ]
  }  tags = local.common_tags
}

So finally the ECS service can be defined:

resource "aws_ecs_service" "influxdb" {
  name             = "influxdb"
  cluster          = aws_ecs_cluster._.id
  task_definition  = aws_ecs_task_definition.influxdb.arn
  desired_count    = 1  launch_type      = "FARGATE"
  platform_version = "1.4.0"   network_configuration {
    security_groups = [aws_security_group.influxdb_access.id]
    assign_public_ip = true
    subnets          = aws_subnet._.*.id
  }  tags           = local.common_tags
}

with the following task definition.

The secret sauce

The important part here is to realize that the platform_version must be set to 1.4.0. While The Terraform documentation states

Defaults to LATEST.

- this is results to 1.3.0— which does not support the efs_volume_configuration section.

resource "aws_ecs_task_definition" "influxdb" {
  family                = "Example-${terraform.workspace}-influxdb"
  container_definitions = data.template_file.influxdb.rendered  task_role_arn      = aws_iam_role.influxdb.arn
  execution_role_arn = aws_iam_role.influxdb_execution.arn  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = var.influxdb_cpu
  memory                   = var.influxdb_memory  volume {
    name = "influxdb-storage"
    efs_volume_configuration {
      file_system_id = aws_efs_file_system.influxdb.id
    }
  }  tags = local.common_tags
}data "template_file" "influxdb" {
  template = file("task-definitions/influxdb.tpl")  vars = {
    cpu    = var.influxdb_cpu
    memory = var.influxdb_memory    region    = var.aws_region
    log_group = aws_cloudwatch_log_group.influxdb.name
  }
}

with the most basic InfluxDB task definition:

[{
  "name": "influxdb",
  "image": "influxdb:1.6",
  "essential": true,  "cpu": ${cpu},
  "memory": ${memory},  "portMappings": [{
    "containerPort": 8086,
    "hostPort": 8086
  }],  "mountPoints": [{
    "containerPath": "/var/lib/influxdb",
    "sourceVolume": "influxdb-storage"
  }],  "logConfiguration": {
    "logDriver": "awslogs",
    "options": {
      "awslogs-group": "${log_group}",
      "awslogs-region": "${region}",
      "awslogs-stream-prefix": "ecs"
    }
  }
}]