SPEEDING UP THE JENKINS ECS DEPLOYMENT

PRE-REQUISITE

The tools and processes for deploying on ECS involve:

• Jenkins
• Script
• Docker
• ECS (Elastic Container Service)

ISSUE

Deployment started taking 12–15mins in DEVELOPER ENVIRONMENT impacting developer productivity.

SOLUTION

Deployment time was reduced from 12–15mins to 2–3mins.

(1) Identified and removed unwanted files

Lot of times we leave unwanted files in GIT, build directory which increases the build size. Used the below command to figure out bulky unwanted files.

Our docker images size was reduced from 1.5 GB to 700 MB.

> du -h -d 1

(2) Docker caching

Another way to save time and make the build faster is by using docker caching. You can COPY bulky static files first and then dynamic files. This allow docker to use previously cached docker image instead of copying it again.

Example.
COPY staging.json /usr/src/app
COPY . /usr/src/app
…
Step 4/7 : COPY staging.json /usr/src/app
 — -> Using cache
 — -> b817d0589b44
Step 5/7 : COPY . /usr/src/app
 — -> 6c798fdcca3d
…

Reference Link — https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#avoid-installing-unnecessary-packages

(3) Removed ‘long sleep’ with ‘short sleep — deployment check’

Instead of putting a ‘long sleep’ and wait for deployment to be over, I replaced it with a do-while loop which checks whether ECS deployment is over or not. If not it sleeps for 5sec and check again.

wait_service_deployment () {
  AWS_ECS_CLUSTER=$1
  SERVICE_NAME=$2
desired_count=`aws ecs describe-services --service "$SERVICE_NAME" --cluster "$AWS_ECS_CLUSTER" --region ap-southeast-1 --output json | jq '.services[].deployments[] | select(.status=="PRIMARY") | .desiredCount'`
while true
  do
    echo -e "\nWaiting for service deployment :: $SERVICE_NAME\n"
    running_count=`aws ecs describe-services --service "$SERVICE_NAME" --cluster "$AWS_ECS_CLUSTER" --region ap-southeast-1 --output json | jq '.services[].deployments[] | select(.status=="PRIMARY") | .runningCount'`
    active_running_count=`aws ecs describe-services --service "$SERVICE_NAME" --cluster "$AWS_ECS_CLUSTER" --region ap-southeast-1 --output json | jq '.services[].deployments[] | select(.status=="ACTIVE") | .runningCount'`
    echo -e "\nRUNNING CONTAINER :: $running_count, DESIRED CONTAINER :: $desired_count\n"
    
    if [[ "$running_count" -eq "$desired_count" && -z "$active_running_count" ]]
    then
      break;
    else
      sleep 5
    fi
  
  done
}