AWS ECS Service Deployment: Azure DevOps Pipelines, Python, Terragrunt and Manifest File
This is a step-by-step guide on deploying an AWS ESC Service to a existing AWS ECS Cluster using Azure DevOps Pipelines, Terragrunt and a Manifest File.
Prerequisites
- AWS Account with existing ECS Fargate Cluster.
- Azure DevOps Service connection for AWS environment.
- Terragrunt bootstrapped (Created Dynamodb lock table and s3 bucket for Terraform state) in AWS Account. You can follow the Terraform state documentation here to configure the state.
- Hello World container image stored in AWS ECR.
Process breakdown
- The pipeline starts by cloning two repositories: ecs-service and ecs-service-manifest.
- Next, the pipeline executes a task to install and configure Terraform and Terragrunt.
- The main pipeline task handles authentication with the AWS Account. Following this, a Python script is executed, which reads the manifest.yml file located in the ecs-service-manifest repository (cloned in the previous step). The script uses the provided AppName parameter value to extract all the keys and values from the manifest file that match the specified AppName.
- The Python script then formats the extracted keys and values to meet the requirements of Terragrunt using the terraform yamldecode command.
- The formatted keys and values are appended to the terragrunt.hcl file in the following location: ecs-service/ecs-service-deployment/live/dev/eu-west-1/ecs-service/terragrunt.hcl.
- Finally, the pipeline executes a terragrunt apply command on the ecs-service/ecs-service-deployment/live/dev/eu-west-1/ecs-service/terragrunt.hcl file, initiating the deployment of the ECS Service to the ECS Fargate Cluster
Step 1
Create two new Azure DevOps repositories:
Follow the steps on the official Microsoft documentation here and create two new repositories ecs-service and ecs-service-manifest.
The ecs-service repository will store the Terraform module, Terragrunt configuration and pipeline files to create and deploy the AWS ECS Service and the ecs-service-manifest repository will store the manifest file that describes the AWS ECS Service.
The full code for the repositories can be found on GitHub.
Step 2
In the ecs-service repository create the following directory structure and files. The directory structure follows the Terragrunt dry structure here.
Below are the contents of the files
- ecs-service-deployment/live/_env/ecs-services.hcl
terraform {
source = "../../../../terraform-modules//ecs-service"
}
- ecs-service-deployment/live/dev/eu-west-1/ecs-service/terragrunt.hcl
include "root" {
path = find_in_parent_folders()
}
include "region" {
path = find_in_parent_folders("region.hcl")
}
include "env" {
path = "${get_terragrunt_dir()}/../../../_env/ecs-service.hcl"
}
- ecs-service-deployment/live/dev/eu-west-1/region.hcl
Update the bucket and dynamodb_table parameters in the region.hcl file to your environment values.
terraform {
extra_arguments "env" {
commands = [
"apply",
"plan",
"import",
"push",
"refresh",
]
arguments = [
"-var", "user=${get_env("USER", "NOT_SET")}",
"-var", "env=${get_env("TF_VAR_env", "dev")}",
"-var", "app_name=${get_env("app_name", "")}",
]
}
}
remote_state {
backend = "s3"
generate = {
path = "backend_generated.tf"
if_exists = "overwrite_terragrunt"
}
config = {
bucket = ""
key = "${get_path_from_repo_root()}/${get_env("app_name", "")}/terraform.tfstate"
region = "eu-west-1"
encrypt = true
dynamodb_table = ""
}
}
generate "provider" {
path = "provider_generated.tf"
if_exists = "overwrite_terragrunt"
contents = <<EOF
provider "aws" {
region = "eu-west-1"
default_tags {
tags = {
AwsEnvironment = title(var.env)
Creator = "${get_env("USER", "NOT_SET")}"
ManagedBy = "terraform"
}
}
}
EOF
}
locals {
env = "dev"
region = "eu-west-1"
azs = [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"]
}
- ecs-service-deployment/live/terragrunt.hcl
terraform {
extra_arguments "retry_lock" {
commands = get_terraform_commands_that_need_locking()
arguments = ["-lock-timeout=60m"]
}
}
generate "versions" {
path = "versions.tf"
if_exists = "overwrite"
contents = <<EOF
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "4.67.0"
}
}
}
EOF
}
## !!
generate "config" {
path = "config_generated.tf"
if_exists = "overwrite_terragrunt"
contents = <<EOF
variable "user" {
type = string
description = "Current user who's executing the plan"
}
variable "env" {
type = string
description = "Environment"
}
EOF
}
- pipelines/pipeline-dev.yml
name: "ECS Deploy Service ${{ parameters.appName }}"
trigger:
- none
pool:
vmImage: ubuntu-latest
parameters:
- name: environmentName
type: string
default: "dev"
values:
- "dev"
- name: appName
type: string
default: ecs application name, e.g. app-origination
- name: regionName
type: string
default: 'eu-west-1'
- name: cicdIAMRoleName
type: string
default: ''
- name: awsAccountID
type: string
default: ''
- name: ServiceConnectionName
type: string
default: ''
- name: azureDevOpsProjectName
type: string
default: ''
- name: branchName
displayName: branchName - ecs-service-manifest
type: string
default: 'main'
stages:
- template: templates/pipeline.yml
parameters:
environmentName: ${{ parameters.environmentName }}
appName: ${{ parameters.appName }}
regionName: ${{ parameters.regionName }}
branchName: ${{ parameters.branchName }}
cicdIAMRoleName: ${{ parameters.cicdIAMRoleName }}
awsAccountID: ${{ parameters.awsAccountID }}
ServiceConnectionName: ${{ parameters.ServiceConnectionName }}
azureDevOpsProjectName: ${{ parameters.azureDevOpsProjectName }}
- pipelines/templates/install-tg-tf.yml
steps:
- script: |
echo "##### INSTALL TERRAFORM #####"
git clone --depth=1 https://github.com/tfutils/tfenv.git ~/.tfenv
echo 'export PATH="$HOME/.tfenv/bin:$PATH"' >> ~/.bash_profile
mkdir -p ~/.local/bin/
. ~/.profile
ln -s ~/.tfenv/bin/* ~/.local/bin
tfenv use 1.4.4
echo "##### INSTALL TERRAGRUNT #####"
wget https://raw.githubusercontent.com/warrensbox/tgswitch/release/install.sh
chmod +x ./install.sh
./install.sh -b ~/.tfenv/bin/
~/.tfenv/bin//tgswitch --bin=/home/vsts/.tfenv/bin/terragrunt 0.47.0
displayName: Terraform & Terragrunt Install
- pipelines/templates/pipeline.yml
parameters:
- name: environmentName
type: string
- name: appName
type: string
- name: regionName
type: string
- name: branchName
type: string
- name: cicdIAMRoleName
type: string
- name: awsAccountID
type: string
- name: ServiceConnectionName
type: string
- name: azureDevOpsProjectName
type: string
stages:
- stage: Deploy
displayName: Deploy ECS Service
jobs:
- job: TerraformApply
steps:
- checkout: git://${{ parameters.azureDevOpsProjectName }}/ecs-service-manifest@refs/heads/${{ parameters.branchName }}
- checkout: self
- template: install-tg-tf.yml
- task: AWSShellScript@1
displayName: DeployService
inputs:
awsCredentials: ${{ parameters.ServiceConnectionName }}
regionName: ${{ parameters.regionName }}
scriptType: "inline"
inlineScript: |
## Set AWS Credentials
set -e
# ls -al
# env | sort | grep DEV
# echo "[+] Pre assume role"
# env | sort | grep AWS | rev
# Connection to the aws account
ACCOUNT_NAME=$(echo '${{ parameters.environmentName }}' | tr '[:lower:]' '[:upper:]')
ACCOUNT_ID="${{ parameters.awsAccountID }}"
echo $ACCOUNT_NAME
OUT=$(aws sts assume-role --role-arn "arn:aws:iam::${ACCOUNT_ID}:role/${{ parameters.cicdIAMRoleName }}" --role-session-name AWSCLI-Session)
export AWS_ACCESS_KEY_ID=$(echo $OUT | jq -r '.Credentials''.AccessKeyId');\
export AWS_SECRET_ACCESS_KEY=$(echo $OUT | jq -r '.Credentials''.SecretAccessKey');\
export AWS_SESSION_TOKEN=$(echo $OUT | jq -r '.Credentials''.SessionToken');
# echo "[+] Post assume role"
# aws sts get-caller-identity
# env | sort | grep AWS | rev
export CI=true
ACCOUNT_ID="$(aws sts get-caller-identity | jq -r '. .Account')"
echo "##### Generate inputs for ecs service #####"
python -m pip install --upgrade pip
python - <<SCRIPT
import yaml
import subprocess
# Load the YAML Manifest file
with open('ecs-service-manifest/manifest.yml') as file:
data = yaml.safe_load(file)
# Define the target app_name
target_app_name = '${{ parameters.appName }}'
# Filter the collection based on the target app_name
filtered_collection = [item for item in data if item.get('app_name') == target_app_name]
# Write the filtered collection to a file
with open('manifest_tmp.yml', 'w') as output_file:
output_file.write(yaml.dump(filtered_collection))
# Run the terraform yamldecode command
cmd = ['terraform', 'console']
input_cmd = 'yamldecode(file("manifest_tmp.yml"))["0"]'
output = subprocess.check_output(cmd, input=input_cmd.encode('utf-8'))
# Format the output string and remove extra brackets
output_string = output.decode('utf-8').strip().strip('{}')
# Append the output string to the file
with open('ecs-service/ecs-service-deployment/live/${{ parameters.environmentName }}/${{ parameters.regionName }}/ecs-service/terragrunt.hcl', 'a') as file:
file.write(f"inputs = {{{output_string}}}\n")
# Confirmation message
print("Inputs generated for terragrunt.hcl")
SCRIPT
cat ecs-service/ecs-service-deployment/live/${{ parameters.environmentName }}/${{ parameters.regionName }}/ecs-service/terragrunt.hcl
echo "##### TERRGRUNT RUN #####"
# Export app_name parameter as global variable
export app_name=$(echo '${{ parameters.appName }}');
cd ecs-service/ecs-service-deployment/live/${{ parameters.environmentName }}/${{ parameters.regionName }}/ecs-service/
~/.tfenv/bin/terragrunt apply -var "aws_account_id=${ACCOUNT_ID}" -var "app_name=${app_name}" --auto-approve --terragrunt-non-interactive -compact-warnings
- ecs-service-deployment/terraform-modules/ecs-service/main.tf
Update the following in the main.tf file. To make testing the service deployed easier, use public subnets.
- subnet_ids
- subnets
- vpc_id
module "ecs_service" {
source = "git::https://github.com/terraform-aws-modules/terraform-aws-ecs.git//modules/service"
create = var.create
cluster_arn = "arn:aws:ecs:${var.region}:${var.aws_account_id}:cluster/${var.cluster_name}"
create_task_definition = var.create_task_definition
network_mode = "awsvpc"
family = "${var.app_name}-family"
name = var.app_name
subnet_ids = [""]
security_group_ids = [module.security_group.security_group_id]
create_security_group = false
deployment_controller = {
minimum_healthy_percent = 100
maximum_percent = 200
type = "ECS"
}
load_balancer = {
service = {
container_name = var.app_name
container_port = var.ecs_container_port
target_group_arn = element(module.alb.target_group_arns, 0)
}
}
container_definitions = {
task_definition = {
readonly_root_filesystem = false
name = var.app_name
image = "${var.aws_account_id}.dkr.ecr.eu-west-1.amazonaws.com/${var.image}:${var.image_version}"
port_mappings = [
{
containerPort = var.ecs_container_port
protocol = "tcp"
hostPort = var.ecs_host_port
}
]
environment = var.env_variables != null ? var.env_variables : []
secrets = var.secret_env_variables != null ? var.secret_env_variables : []
cpu = var.ecs_container_cpu
memory = var.ecs_container_memory
}
}
}
module "security_group" {
source = "git::https://github.com/terraform-aws-modules/terraform-aws-security-group.git//"
name = "${var.app_name}-sg"
description = "Service security group"
vpc_id = [""]
ingress_rules = ["http-${var.ecs_container_port}-tcp"]
ingress_cidr_blocks = var.ingress_cidr_blocks
egress_rules = ["all-all"]
egress_cidr_blocks = var.egress_cidr_blocks
}
module "alb" {
source = "git::https://github.com/terraform-aws-modules/terraform-aws-alb.git//"
name = "${var.app_name}-alb"
create_security_group = false
load_balancer_type = "application"
vpc_id = ""
subnets = [""]
security_groups = [module.security_group.security_group_id]
http_tcp_listeners = [
{
port = var.ecs_container_port
protocol = "HTTP"
target_group_index = 0
},
]
target_groups = [
{
name = "${var.app_name}-alb-tg"
backend_protocol = "HTTP"
backend_port = var.ecs_container_port
target_type = "ip"
},
]
}
- ecs-service-deployment/terraform-modules/ecs-service/variables.tf
variable "create" {
default = true
}
variable "create_task_definition" {
default = true
}
variable "ecs_container_port" {
type = any
}
variable "ecs_host_port" {
type = number
}
variable "ecs_container_cpu" {
type = string
}
variable "ecs_container_memory" {
type = string
}
variable "image" {
type = string
}
variable "image_version" {
type = string
}
variable "app_name" {
type = string
}
variable "region" {
type = string
default = "eu-west-1"
}
variable "aws_account_id" {
type = string
}
variable "cluster_name" {
type = string
}
variable "env_variables" {
type = list(map(string))
default = []
}
variable "secret_env_variables" {
type = list(map(string))
default = []
}
Step 3
In the ecs-service-manifest repository create a manifest.yml file.
/manifest.yml
Update the values for app_name, cluster_name and image_name in the manifest.yml file. The app_name value must match the appName parameter value used in the pipeline.
It is expected that the container image is stored in AWS ECR in your AWS account. The image used for testing is a Nginx web server that displays Hello World.
- app_name: <app_name>
cluster_name: <cluster_name>
image: <image_name>
image_version: 1
ecs_container_port: 80
ecs_host_port: 80
ecs_container_cpu: 256
ecs_container_memory: 512
env_variables:
- name: ENV_VAR1
value: example1
- name: ENV_VAR2
value: example2
- name: ENV_VAR3
value: example3
- name: ENV_VAR4
value: example4
Step 4
Create the new Azure DevOps pipeline using the Azure DevOps portal.
- Navigate to Pipelines
- Click on New Pipeline
- Select Azure Repos Git when asked “Where is your code?”
- Select the ecs-service repository
- Select Existing Azure Pipelines YAML file
- For Branch select main and for Path, set the path to pipelines/pipeline-dev.yml
- Review the pipeline code and click the Run button dropdown and select save.
- Next click on Run pipeline. Fill in the required parameters and click Run. Take not of the appName parameter which should match the appName in the manifest.yml file in the ecs-service-manifest repo.
- If the pipeline ran successfully, you should see a similar page as below with all steps completed successfully.
Step 5
Next, navigate to EC2 > Load Balancers in the AWS Console for the AWS Account that you deployed the ECS Service to.
You will see a new load balancer named hello-world-alb.
Open the DNS name of the load balancer in a new browser tab
If the ECS Service deployment was successful, you will see a similar page as below depending on the image used for the ECS Service.
You can also view the ECS Service deployed on the ECS Cluster to make sure the ECS Service deployed and is up and running.
Clean up
To destroy all of the infrastructure created, run a terragrunt destroy in the ecs-service/ecs-service-deployment/live/dev/eu-west-1/ecs-service/ directory. You can do this by running the command in your local repo of ecs-service.
cd ecs-service/ecs-service-deployment/live/dev/eu-west-1/ecs-service/
terragrunt apply --destroy
Final thoughts
There are still some improvements that can be made to this method of deployment. Watch this space to view updates in future to this deployment method.