Deploy ASP.NET Core App to AWS EB Running a Linux, using Azure DevOps — Part 2

12 min readNov 7, 2018

A single-container, dockerized web app is chosen to ease the deployment to Linux host.

1. Story

Currently we have been using a straightforward source control flow using git. After pulling source from release/stable branch to make new feature or bug fix in new branch, developer will build and deploy the source code on local/remote computer, where on each task, developer will execute a bunch of build script (npm restore, dotnet test, etc). Some of the project use docker to containerize app so we also need to build image and push the image to remote registry, copying and archiving the output/compiled source, then upload it to host server either using FTP or SSH. A bit adjustment moments later and the app is up.

This tedious deployment task at least require a half developer-hour per app version release (from dev to production). Currently we have various web apps that each may have 1 version release per day. Suppose there are 4 apps, we must allocate at least ~2 developer-hours already per day.

Fortunately, there is a way to shorten the deployment task by removing part of human tasks. Instead of human, a machine (build agent) will take the charge for executing build script. This build agent will execute all necessary deployment task defined by us, automatically every time there is a code changes on repository. This mechanism allows us to reduce almost 80% time required in deployment task and increase consistency across releases.

There are tools available for free to achieve this, such as Jenkins, GitLab CI, Travis CI and Azure DevOps. I prefer to use Azure DevOps, because it is already packed with end-to-end work collaboration service such as repository and scrum board to support your DevOps culture, and it has feature that is critically needed, that’s called ‘Path Filter’. This feature allows the trigger to only listen for specific folder inside repository. For example, if we have 3 web app project folders called A, B, C in single repository, we can set the trigger to only listen for code changes on folder A. I barely find these benefits integrated in other tools.

Azure DevOps packed with end-to-end work collaboration service such as repository, scrum board, CI/CD

That’s story behind why I use Azure DevOps, for reason why I use a dockerized web app architecture, please read part 1 of this article. In a nutshell, the reason is to minimize cost as low as possible for anyone who already experienced with .NET.

In this article, I want to share how we use Azure DevOps to deploy ASP.NET Core Web App to AWS EB. Feel free to substitute one of the stacks I use, most of the feature should have its equivalent in other tool. For example, for repository instead of using the built-in feature from Azure DevOps, you can use GitLab, etc.

2. What This Article Cover

Use docker preconfigured platform of AWS EB instance running Amazon Linux, to host dockerized ASP.NET Core site [Part 1 — Previous article]
Enable HTTPS using AWS EB with custom domain and AWS Certificate Manager [Part 1 — Previous article]
Implement CI/CD using Azure DevOps with a Linux user agent [Part 2 — You’re here]
Auto redirection from non-HTTPS/naked URL to fully qualified HTTPS URL [Part 2 — You’re here]

3. Technology Stacks

Web App: ASP.NET Core MVC, written in C#
Unit Test: xUnit
Host OS: Linux
Web Server: Nginx with Kestrel as reverse proxy
Containerization: Docker
IaaS: AWS EB (Elastic Beanstalk), Classic LB (Load Balancer), ECR (Elastic Container Registry), ACM (AWS Certificate Manager)
DevOps tools: Azure DevOps

4. Prerequisites

Azure DevOps Account
AWS Account
Basic knowledge of ASP.NET
Basic knowledge of cloud infrastructure

5. Hands On

You can always jump to step that you want to explore. To give a big picture, this ‘hands on’ is structured as below:

Step 1–5: Preparing the web app cloud stack using AWS [Part 1 — Previous article]

Step 6–9: Preparing Azure DevOps CI/CD pipeline [Part 2 — You’re here]

Step 10–11: Develop web app using ASP.NET Core and deploy it using CI/CD pipeline [Part 2 — You’re here]

Let’s continue to step 6, shall we?

Step 6. Create EC2 instance to host build agent

Build agent is just like a software that will run build script that you define to build and publish your app project. So instead of you, build agent will take in charge. To start using them, we need to give them a host (machine) that is exposed to internet/VPC and has ability to run script you define just like you run your script on your development computer. This build agent support Windows, Linux and Mac. In this tutorial, we are going to use EC2 instance running Ubuntu as a host of our build agent.

Create EC2 instance from Amazon Machine Image (AMI) using launch wizard. There are range of EC2 instance type with various performance. For a starter project, we can use the free tier which is a t2.micro.

t2.micro instance type

Go to step for configuring storage capacity. I suggest to use 50GB because we have to install a few build tools required to deploy our site. Launch the instance. It will take a few minutes to get your instance ready.

Meanwhile, you can setup your project in Azure DevOps. Go to https://azure.microsoft.com/en-us/services/devops/ and create your account or sign in.

Once your EC2 instance is ready, you have to install Azure DevOps build agent by running SSH remote connection to your instance. For this step you can follow the steps as instructed on official documentation here: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-linux?view=vsts. The instruction is really clear and easy to follow.

Step 7. Install build tools

After build agent has been installed, then we need to install required build tools on EC2 instance, using apt-get as instructed on the official documentation. Here is the list:

· .NET Core SDK: to build our ASP.NET Core Web App project. (Tutorial: https://www.microsoft.com/net/download/linux-package-manager/ubuntu16-04/sdk-current)

· Docker: to build & push image to AWS ECR (Tutorial: https://docs.docker.com/install/linux/docker-ce/ubuntu/)

· Git: to fetch source code (Tutorial: https://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-16-04)

· AWS CLI: to setup AWS credential needed to access AWS ECR from build machine (Tutorial: https://docs.aws.amazon.com/cli/latest/userguide/awscli-install-linux.html)

Then check if each tools have been installed properly, by executing their command to see its version (ex: dotnet --version to check dotnet sdk version).

Then check if your build agent has been displayed in Azure DevOps UI as below:

Your build agent should be located in Private pools with the name you provided during installation. You can also notice some tools have been displayed in capabilities list. I have no idea why dotnet is not displayed there but it is working properly.

Step 8. Setup repository & service connection

Initialize new repository for your project

Choose either to clone, push or import repository. Make sure that you have setup git credentials in your local computer so that you can have access to your repository. There is also ‘Clone in VS Code’ button to ease your local repository initialization if you already use VS Code.

Next step, you want to add service connection. This tell Azure DevOps that your build agent requires access to external service like AWS and Docker. This access is defined with API key ID & secret that you can get in its service.

To get AWS service connection, go to AWS IAM and grant roles to your desired user (it is best to use a programmatic access user, instead of creating new member in your AWS organization). Then you will get Key ID & secret to be passed to service connection.

Setup Docker Registry service connection

For docker registry you can obtain the ID and password by executing the login command in your computer, that you get in Step 5. Copy and paste them in above form.

Step 9. Configure Build Pipeline

a. Setup trigger

First you have to select your source repository. for example, I use built-in repository that is Azure Repos Git. Then choose your repo branch. This will be used as a source when trigger happens once the source changes on the configured branch that we will setup on few steps later.

Next, you want to choose pipeline template. In this example, I choose empty job to design the pipeline from scratch via GUI. If you prefer to design them using script, there is YAML option that allows you to do just that.

Next go to Triggers tab, and here you want to select branch that will trigger the pipeline when the source on that branch is changed. You may also want to filter the path inside the branch to listen only to changes on configured path. For example, if you are using monorepo where there is A,B,C projects in single repository, you may want to trigger build of project A only if the changes happen in A project. To do so, you have to exclude all path, then include path to A project.

Notice that I use 2 path filters, means that build tasks will ignore all code changes in root path of repository except for path /YourAspProjectName.

Next go to Tasks tab, and here you want to choose Agent pool that you have configured and verified in step 7.

Next go to Get sources task, verify that the branch match with the branch you configured on Step 9a.

Next go to first Phase, and here you want to design all task in this pipeline, start from dotnet restore. Click on ‘+’ icon to add new task.

As you see that in this tutorial, our repository is still empty but in this step we are providing the contents of repository, this is intended by purpose, so that in last step where we create the project, we just need to push them to repository to see all task in this pipeline ready to takes the charge for deploying the project into our host explained in part 1 of this article.

b. Dotnet restore

Dotnet restore is task used to restore all dependencies required by your asp.net project to run.

c. Dotnet build

Dotnet build is task that you used to build the source. Here I add ‘configuration’ argument that will pass a value from ‘BuildConfiguration’ variable configured in Variables tab.

d. Dotnet test

Dotnet test is used to run unit test configured in your test project. There is plenty frameworks with tutorial here https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-dotnet-test

e. Dotnet publish

Upon successful test, publish (compile) the project to produce executable binary (output)

Dotnet publish is used to compile and release the binary output (.dll file) of the project that is used later by docker to run the web.

f. Build image

Next, you want to create task to build docker image.

Build docker image from the Dockerfile that define how to run the output of your compiled asp.net project

Here I use version 1 of the docker task. Replace image name with your repository url & image tag that you get in step 5. I also add arguments — rm — build-arg certPassword=$(Build.BuildId) to pass certificate password to dockerfile that we created later on step 10. I know that I have decided to use HTTPS termination on the LB, so adding this argument is actually an optional. Just in case we decide to remove HTTPS termination on the LB, we just need to open port 443 of EB instance without the need to change our dockerfile.

g. Push image

Login to AWS ECR and push the image to registry

In this example, I use command line task to run the script that contains command to push the image to AWS ECR. Notice that, before I run the ‘docker push’ command, first I run command ‘eval …’ that we get from previous article part, to login to AWS ECR.

In this step, our docker image will be ready to be pulled and run by AWS EB.

h. Copy artifact

We can run application in EB with single container environment, by specifying Dockerrun.aws.json. If you are new to EB, please read the documentation here.

Artifact is one of deliverable from build task, that describe how to run an application. In this example, this artifact will be used by AWS EB to run the application, and it only contains 2 type of files, first is the Dockerrun.aws.json file that describe how the EB can run the application inside docker. Second is all files inside .ebextensions folder, that describe how the web server app like Nginx should behave (ex: implement HTTPS redirection rule). I recommend you to read this tutorial from AWS: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/docker-singlecontainer-deploy.html and https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_image.html, to understand how project is structured to be able to be deployed on AWS EB.

Dockerrun.aws.json that we will write on few steps later, will describe how to pull the docker image from AWS ECR that has been configured in step 9.g and run it on AWS EB.

i. Archive artifact

An archived artifact is needed by AWS EB, so we have to archive them. Uncheck the ‘prepend root folder’ checkbox to make content placed on the root of the archive, instead of placed inside the project folder.

j. Upload to AWS S3

AWS EB can only run an application that is uploaded from local computer or AWS S3. The latter is used as it is easier to be run by the build agent. So AWS S3 is chosen as the source of the archived artifact.

k. Deploy to AWS EB

Next, we want the build agent to upload the archived artifact that is stored on AWS S3 to AWS EB using the above setting.

Step 10. Create ASP.NET Core Web App project

We need to structure the project so they contains .ebextensions folder and Dockerrun.aws.json.

Initialize local repository and set the origin to remote repository (in this example I use Azure Repo). Go to working directory and create new ASP.NET Core MVC project using dotnet CLI, with this command: dotnet new mvc -o YourAspProjectName. Verify that project is created and can be run locally.

Create a Dockerrun.aws.json file then copy this value. This file will be used by EB instance so that it can understand how to run your site.

Replace field Image.Name with the image repo URL from AWS ECR, including the tag name. We are using port 80 of container to receive incoming request from port 80 of EB instance. Because we have used HTTPS termination on the load balancer, we don’t need to expose port 443 of both EB instance or container.

Next we want to enable auto HTTPS redirection. To do so, create .ebextensions folder in project root, inside it, create a file https-redirect-docker.config and copy this value. This file is an NGINX configuration that will make the load balancer to route incoming request as below.

Then create new Test project in working directory, with this command: dotnet new test -o YourTestProjectName. Verify that test project is created and can be run locally.

Step 11. Push project to remote repository & let the pipeline deploy your fresh site

Run git commit & push. The build pipeline should start in a few seconds. You can check deployment progress in Azure DevOps UI. After build is successful, open some browser and type your site’s URL. That’s it.

If you have any question, don’t hesitate to ask in the comment below. I would love to hear any suggestion to improve this article. Don’t forget to clap if you think this article is worth to share.