Deploying a Multi-Service Application into Multiple Environments using AWS Copilot

AWS CoPilot | DevOps | CICD

Deploying applications to the cloud can be complex, especially when dealing with multiple services and environments. AWS Copilot, a command-line tool by Amazon Web Services (AWS), simplifies this process by providing an easy way to define, develop, deploy and operate containerised applications in AWS ECS, Fargate.

Prerequisites

As perquisites, I assume you have following in place:

1. An AWS Account
2. AWS CLI installed
3. Docker installed
4. AWS Copilot CLI. If not follow https://aws.github.io/copilot-cli/en/docs/getting-started/install/

Let’s get started! 🚀

You can use AWS Copilot to deploy different types of services like Load Balanced Web Service, Backend Service, Request-Driven Web Service, Scheduled Job, Static Site and Worker Service. I am going to focus on deploying an application with 2 Load Balanced Web Services.

Domain

When deploying a Load Balanced Web Service, there are multiple ways we can go about the domain of the services. I am mentioning this at the beginning since you need to initiate the Copilot app the right way that suits your requirement.

• Use an app associated root domain (which you manage in the same AWS account) and let Copilot to manage subdomains for services and environments. Your service urls will be in ${SvcName}.${EnvName}.${AppName}.${DomainName} format
• Use an app associated root domain and customise service domains as you need
• Import SSL certificates if you manage the DNS somewhere other than the AWS account which you are deploying the Copilot services

📖 This is the official document on the above domain options!

App code and Copilot manifests

Main thing to consider is the 2 ways we can use to build the services. Either we can use a Dockerfile and build the service image as a part of Copilot flow or we can use an already built image.

Initiating Copilot app (storing Copilot manifests) within the app repository would be helpful if you are building the Docker image providing the Dockerfile to Copilot. This is suitable for monorepo setups, where all apps/services are in the same repo.

If you are building the Docker image as a part of a separate CI flow, you can provide the docker image to Copilot and store Copilot manifests either within the app code repo or in separate repo.

For this example, we are going to use the first option for app domain and let Copilot create hosted zones, subdomains, SSL certificates under the hood and provide Dockerfile to Copilot to build the image.

Step 0: Sample app repo

Clone the sample application git repository.

git clone https://github.com/ShanikaEdiriweera/aws-copilot-example.git

This is optional. You can start start with a fresh directory and add your own services source code (simple index.html files are used in the sample) and Dockerfiles according to the following structure. Copilot manifests will be generated when you follow the guide.

Directory structure of sample app

Step 1: Init app

Initialise the Copilot application with root domain

copilot app init --domain your-domain.com

You will see the following output creating CloudFormation stack test-app-infrastructure-roles (see the roles and DNS hosted zone created) and generating copilot directory. Note that if you started from scratch you will be prompted for an app name.

Your workspace is registered to application test-app.
✔ Proposing infrastructure changes for stack test-app-infrastructure-roles
- Creating the infrastructure for stack test-app-infrastructure-roles                           [create complete]  [89.9s]
  - A StackSet admin role assumed by CloudFormation to manage regional stacks                   [create complete]  [17.1s]
  - Add NS records to delegate responsibility to the test-app.your-domain.com subdomain          [create complete]  [34.1s]
  - A hosted zone for test-app.your-domain.com                                                   [create complete]  [43.9s]
  - A DNS delegation role to allow accounts: 845688234888 to manage your domain                 [create complete]  [17.7s]
  - An IAM role assumed by the admin role to create ECR repositories, KMS keys, and S3 buckets  [create complete]  [34.5s]
✔ The directory copilot will hold service manifests for application test-app.

Copilot will create a SSM parameter to hold the application metadata to uniquely identify the created application for the future CLI interactions.

Step 2: Deploy first service

Init the first service and deploy into test environment

copilot init

Follow the prompts to configure your application. You’ll need to choose the service type (Load Balanced Web Service, Backend Service, Scheduled Job), application name, and other details.

Copilot will identify existing service manifest file or generate one. Then deploy multiple CloudFormation stacks for the infrastructure, environment and the service as in above and below images. Then it will build your container image from the Dockerfile you specified and push to ECR.

Now you can access your service1 deployed in test environment at https://service1.test.test-app.your-domain.com/

service1 in test

Step 3: Deploy second service

Lets initialise and deploy another service!

copilot svc init --name service2
copilot svc deploy --name service2 --env test

At the end of deploying the CFN stacks, build container image and deploying into test environment you will be able to access service2 in test environment at https://service2.test.test-app.your-domain.com over the internet.

Forgive me me for the ugly interface!

Step 4: Deploy second environment

Lets initialise and deploy our second environment dev and then deploy the 2 services into the dev environment.

copilot env init --name dev
copilot env deploy --name dev

Note that I used default configuration to allow AWS Copilot to create a new VPC and subnets. But have the option to import your existing VPC and subnets.

copilot init env

copilot env deploy — name dev

copilot svc deploy --name service1 --env dev
copilot svc deploy --name service2 --env dev

After deploying the 2 services in the dev environment you would be able to access them at https://service1.dev.test-app.your-domain.com and at https://service2.dev.test-app.your-domain.com

TADA! 🦸🏼‍♂

Now you have 2 services deployed in 2 environments just within 30mins! How cool is that!

Next lets look at how we can automate the deployment of changes to the services.

S️tep 5: Deploy deployment pipeline

Lets initiate a pipeline and deploy it.

copilot pipeline init \
-u https://github.com/ShanikaEdiriweera/aws-copilot-example \
-b main \
-n test-app-pipeline \
-p Workloads \
-e test,dev

copilot pipeline deploy

This will initiate a pipeline which will be looking at changes to the main branch of https://github.com/ShanikaEdiriweera/aws-copilot-example (you can replace with your code repository) and deploy the workload changes to the 2 environments test and dev .

When deploying you will need to provide access to the Github repository from AWS connections.

Provide access!

After deployment is completed you will get a super cool pipeline in AWS CodePipeline!

Step 6: Clean up

Lets delete all the resources Copilot set up for your application, including your ECS Service and the ECR Repository.

$ copilot app delete

Support

AWS Copilot is an open source tool and is backed by a great passionate dev community. If you run into any issue you can get support by creating an issue in official AWS Copilot GH repository or you can get help from ever so supportive fellow Copilots in the chat on gitter! 👨🏽‍✈️

Conclusion

AWS Copilot is a great tool which can be used for quickly deploy your service based application into AWS ECS Fargate, deploy multiple environments and deploy deployment pipelines. It currently has support to add secrets with AWS SSM and to storage with S3, DynamoDB and EFS.

And you should not worry if there is anything you need to implement, which is currently supported by Copilot, you can use the overrides feature to override the CFTs.

AWS Copilot abstracts the underlying complexities, allowing you to focus on your code and delivering value. In a nutshell, AWS Copilot reduces a bulk of CloudFormation template code into few manifest files!

Let me know how it goes and anything I missed or need improvement in!

Happy Copiloting! 👨🏽‍✈️🚀

Cheers! 🍻

Reference

1. AWS Copilot Official Documentation — https://aws.github.io/copilot-cli/en/docs/overview/
2. AWS Copilot GitHub Repository — https://github.com/aws/copilot-cli
3. Sample Application Repository — https://github.com/ShanikaEdiriweera/aws-copilot-example