Run MicroServices on AWS Serverless Platform (AWS Fargate)
Introduction
The serverless architecture enables you to shift more of the operational responsibilities to the cloud service provider. The user does not need to worry about the infrastructure management and thus can focus more on building and managing their application life cycle. The fault tolerance and availability aspect also will be taken cared by the cloud service provider.
AWS Fargate is the serverless platform from AWS. It eliminate the need to manage the cluster or EC2 instances, rather lets the user focus on managing the application lifecycle.
Deployment Architecture
As per the Well-Architechted guide lines, we’ll use three private subnets, each on separate AZs to deploy the containers. Three public subnets will be used to provision the ALB, and NATs if needed. Provision the endpoints to access the managed service APIs internally as shown above.
Dev environment and Tools setup
We’ll use the cloud9 IDE for this demo. Setup the cloud9 IDE using the instructions here.
Install ecs-cli, other utilities
$ sudo curl -so /usr/local/bin/ecs-cli https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-linux-amd64-latest $ sudo chmod +x /usr/local/bin/ecs-cli$ sudo yum -y install jq gettext
Configure defaults
$ export AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region)
$ echo "export AWS_REGION=${AWS_REGION}" >> ~/.bash_profile
$ aws configure set default.region ${AWS_REGION}
$ aws configure get default.region
Clone Repos
Clone the following repos to get the sample code and templates used in the demo
$ git clone https://github.com/hmanikkothu/ecs-platform.git$ git clone https://github.com/hmanikkothu/ecs-nodejs-backend$ git clone https://github.com/hmanikkothu/ecs-nodejs-frontend
Build VPC, ECS Cluster, ALB
AWS CloudFormation provides a way to use simple text/yaml file to model and provision resources in an automated and repeatable way. In this example we’ll use cloud formation templates for creating the VPC/Subnets, Cluster and ALBs
Build VPC, Cluster
$ cd ecs-platform$ aws cloudformation deploy --stack-name ecs-nodejs-dev --template-file cluster-fargate-vpc.yml --capabilities CAPABILITY_IAM
The cloud formation template cluster-fargate-vpc.yml defines the VPC, Subnets, IG, NATs, Security Groups, Route Tables and the Cluster.
Build ALB
$ aws cloudformation deploy --stack-name ecs-nodejs-dev-alb --template-file alb-external.yml
The cloud formation template ‘alb-external.yml’ create an ALB, associate it with all three public subnets, and creates a dummy target group which will be used to load balance the ECS tasks later.
Deploy Backend Service Container
Set env variables
$ export clustername=$(aws cloudformation describe-stacks --stack-name ecs-nodejs-dev --query 'Stacks[0].Outputs[?OutputKey==`ClusterName`].OutputValue' --output text)$ export target_group_arn=$(aws cloudformation describe-stack-resources --stack-name ecs-nodejs-dev-alb | jq -r '.[][] | select(.ResourceType=="AWS::ElasticLoadBalancingV2::TargetGroup").PhysicalResourceId')$ export vpc=$(aws cloudformation describe-stacks --stack-name ecs-nodejs-dev --query 'Stacks[0].Outputs[?OutputKey==`VpcId`].OutputValue' --output text)$ export ecsTaskExecutionRole=$(aws cloudformation describe-stacks --stack-name ecs-nodejs-dev --query 'Stacks[0].Outputs[?OutputKey==`ECSTaskExecutionRole`].OutputValue' --output text)$ export subnet_1=$(aws cloudformation describe-stacks --stack-name ecs-nodejs-dev --query 'Stacks[0].Outputs[?OutputKey==`PrivateSubnetOne`].OutputValue' --output text)$ export subnet_2=$(aws cloudformation describe-stacks --stack-name ecs-nodejs-dev --query 'Stacks[0].Outputs[?OutputKey==`PrivateSubnetTwo`].OutputValue' --output text)$ export subnet_3=$(aws cloudformation describe-stacks --stack-name ecs-nodejs-dev --query 'Stacks[0].Outputs[?OutputKey==`PrivateSubnetThree`].OutputValue' --output text)$ export security_group=$(aws cloudformation describe-stacks --stack-name ecs-nodejs-dev --query 'Stacks[0].Outputs[?OutputKey==`ContainerSecurityGroup`].OutputValue' --output text)
Configure default lauch_type and cluster for ecs-cli
$ ecs-cli configure --region $AWS_REGION --cluster $clustername --default-launch-type FARGATE --config-name ecs-nodejs-dev
This creates an entry in ‘/home/ec2-user/.ecs/config’ file with the given params.
Deploy backend service
ecs-cli looks for two files in the current folder to load task definitions and run params — ‘ecs-params.yml’ and ‘docker-compose.yml’. Lets take a look at the content of both.
docker-compose.yml — configures the image and logging info.
version: "3"
services:
ecs-nodejs-backend:
image: hmanikkothu/ecs-nodejs-backend
ports:
- "3000:3000"
logging:
driver: awslogs
options:
awslogs-group: ecs-nodejs-backend
awslogs-region: ${AWS_REGION}
awslogs-stream-prefix: ecs-nodejs-backend
ecs-params.yml (templated from ecs-params.yml.template) — Contains task definition, network configuration and service discovery configurations.
version: 1
task_definition:
task_execution_role: arn:aws:iam::1234567890:role/ecs-nodejs-dev-ECSTaskExecutionRole-xxxxxxxxxx
ecs_network_mode: awsvpc
task_size:
mem_limit: 0.5GB
cpu_limit: 256
run_params:
network_configuration:
awsvpc_configuration:
subnets:
- "subnet-081f78af998xxxxxx"
- "subnet-0804f7fc188xxxxxx"
- "subnet-03e3de40966xxxxxx"
security_groups:
- "sg-0321f3d90ea6xxxxxx"
assign_public_ip: DISABLED
service_discovery:
container_name: ecs-nodejs-backend
service_discovery_service:
name: ecs-nodejs-backend
description: string
Run the following commands to deploy
$ cd ecs-nodejs-backend/$ envsubst < ecs-params.yml.template >ecs-params.yml$ ecs-cli compose --project-name ecs-nodejs-dev-backend service up \
--create-log-groups \
--private-dns-namespace service \
--enable-service-discovery \
--cluster-config ecs-nodejs-dev \
--vpc $vpc
envsubst templates the ecs-params.yml with the current values.
The above command also creates private dns namespace for service discovery, and log group in cloudwatch logs. The service can be accessed by other microservices using ‘http://ecs-nodejs-backend.service:<port>’ end point. This will be used in our frontend container as an env variable.
Deploy Frontend Service Container
$ cd /ecs-nodejs-frontend/$ envsubst < ecs-params.yml.template >ecs-params.yml$ ecs-cli compose --project-name ecs-nodejs-dev-frontend service up \
--create-log-groups \
--target-group-arn $target_group_arn \
--private-dns-namespace service \
--enable-service-discovery \
--container-name ecs-nodejs-frontend \
--container-port 3000 \
--cluster-config ecs-nodejs-dev \
--vpc $vpc
Note that the command uses ‘target-group-arn’ param, which will ensure the containers are associated with the ALB target group.
Also, note that the container definition in ‘docker-compose.yml’ sets the backend url as an environment variable.
version: "3"
services:
ecs-nodejs-frontend:
environment:
- BACKEND_SVC_URL=http://ecs-nodejs-backend.service:3000
image: hmanikkothu/ecs-nodejs-frontend
ports:
- "3000:3000"
logging:
driver: awslogs
options:
awslogs-group: ecs-nodejs-frontend
awslogs-region: ${AWS_REGION}
awslogs-stream-prefix: ecs-nodejs-frontend
Test
Get the ALB external usr as follows
$ aws cloudformation describe-stacks --stack-name ecs-nodejs-dev-alb --query 'Stacks[0].Outputs[?OutputKey==`ExternalUrl`].OutputValue' --output text------- output ------http://ecs-n-Publi-2FHCPHJK7BWT-1234567890.us-east-2.elb.amazonaws.com
Open the url in browser, validate the response from frontend service.