Creating a CI/CD Environment and Deploying Containers in the Cloud
In this article, we’ll be setting up an environment for automated building, testing, and deployment, using containers and services hosted in the cloud.
I’ll be assuming that you already have some knowledge of what CI/CD is, and you’re looking to implement its ideas on your project. Furthermore you must have some previous knowledge of Docker, some of basics AWS services, and software engineering in general.
Given we’re working with Docker, the technology you’re using is not so important. But for this tutorial, we’ll be using Flask for its simplicity.
First things first, let me present to you our tools. For our code repository, we’ll be using Github. We need a CI tool to build, run the tests, and deploy our application (if everything goes well). And for that, we’re taking Travis-CI. For last, we need somewhere to host our application and, since we’re talking Docker, let’s use AWS Elastic Container Service (ECS).
Now let’s see how the workflow works:
Every time a commit or a merge is done to a branch, Travis will be triggered and run a list of instructions to build and run our tests. If the build is done successfully and every test passes, Travis will push our Docker image to Docker Hub and trigger an update event on ECS, telling to our cluster that has a new image version to be downloaded.
I’ve made a code template to get things easier to set up. Feel free to use it:
This template is aimed to help you set up a CI/CD development environment on the cloud! We will be using the following…
So let’s get our hand dirty and do some work. This is how our source tree looks like:
To make things easier, our Python application consists of two files.
main.py contains the Flask code and
test.py has the unit test. Since its simplicity, the unit test verifies what a function that returns a sum of two numbers (it has no useful purpose).
Our Python needs a txt file telling what dependencies it needs and its versions, so our requirements.txt will look like this:
To get things started with Docker, first, you need an account on Docker Hub, go ahead and create if you don’t have one yet: https://hub.docker.com/. If you not familiar with it, its a Docker image repository and it’s where ECS will be getting the latest version of our application. (ps: make sure to remember your username).
Now let’s build a Dockerfile and a docker-compose for our Python code and upload it to Docker Hub.
docker-compose.yml make sure to replace “YOUR_USER” for your Docker Hub username on line 8. After the slash, you can choose the name of your application.
To upload the image, let’s build and push it to Docker Hub with these terminal commands:
$ docker-compose build --pull$ docker-compose push
Go to your homepage, you will see your Docker image there.
For our CI tool, log in Travis website with your Github account and sync your repositories. Every time you commit your code to Github, it will trigger a new build (it shouldn’t work by now, since we don’t have an ECS cluster).
Let’s create a
.travis.ymlfile, telling all instructions it must follow to build, test, and deploy our application. The instructions work by jobs, first, you have to specify its settings (such a distro, language compiler, if sudo is needed), if not specified, it will use the default ones. Then you can execute shell commands on which job.
Travis has the following list of jobs, that will be executed in this exact order (you don’t have to use all of them):
This is our
Now that we have all our files set up, let’s create an ECS cluster. AWS is a very complex host service, so I highly recommend you READ THE DOCS (at least ECS ones) since AWS offers a bunch of services and they can charge you for using their services. But for this example, the free tier covers us.
ECS cluster resources depend on your application needs. I’ll show you how to set up an environment good enough for our Flask application. But keep in mind you should do a custom configuration for your project.
Go to AWS website and log to your AWS Management Console
Search for “ECS”
Click on “get started”.
Select the custom Container definition and hit “configure”.
Choose a name for your Container. On Image use this format: “Docker Hub user / your image name : latest”, or just get it from your image repository on Docker Hub. Mine for example is:
leozz37/cicd-example:latest.Map your ports (for Flask we’re using 8000). Then click Update.
Select “Application Load Balancer” and click Next.
Chose a Cluster Name and click Next.
Review all your configurations and click Create.
Wait for your Cluster to get done. If you don’t remember your Service and Cluster name, you can check them here later.
Now we have to set up our Travis environment variables. Go to your project on Travis and go to settings. On Environment Variables, you need to set up yours this way:
Make sure to have every variable set up to their equivalent values (the names are self-explanatory).
Now everything is set up, on AWS go to load balancer, and on Basic Configuration, copy your DNS Name and search for it on your browser with the port 8000 (or to the port you set up).
Our code is running!
Now let’s change something on our code and commit it to witness the magic.
Go to Travis and watch it doing all the hard work.
After building and testing, if everything goes well, Travis will deploy our image to Docker Hub and tell ECS that there’s a new version of it.
If you refresh your browser, you should see your update.
And that’s it! You just created a CI/CD environment using (kinda) free tools and everything hosted on the cloud!