Building a CI/CD pipeline with Travis-CI, Docker and AWS in 9 steps

Published in

Paul Zhao Projects

13 min readAug 24, 2020

In this project, we are going to build up a CI/CD Pipeline for free using with TravisCI and Terraform to automate the creation of the infrastructure.

What is CI/ CD?

Successful Continuous Integration means new code changes to an app are regularly built, tested, and merged to a shared repository. It’s a solution to the problem of having too many branches of an app in development at once that might conflict with each other.

Continuous delivery usually means a developer’s changes to an application are automatically bug tested and uploaded to a repository (like GitHub or a container registry), where they can then be deployed to a live production environment by the operations team. It’s an answer to the problem of poor visibility and communication between dev and business teams. To that end, the purpose of continuous delivery is to ensure that it takes minimal effort to deploy new code.

Continuous deployment (the other possible “CD”) can refer to automatically releasing a developer’s changes from the repository to production, where it is usable by customers. It addresses the problem of overloading operations teams with manual processes that slow down app delivery. It builds on the benefits of continuous delivery by automating the next stage in the pipeline.

Install Docker (Mac user)

Install Homebrew

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install Docker using Hombrew

$ brew install bash-completion
$ brew cask install docker
$ brew install kubectl
$ brew cask install minikube

To verify installation

$ docker version 
Client: Docker Engine - Community
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:41:33 2020
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.12
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       48a66213fe
  Built:            Mon Jun 22 15:49:27 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683$ docker-compose version
docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.1g  21 Apr 2020$ docker-machine --version
docker-machine version 0.16.0, build 702c267f$ kubectl version --client
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-21T14:51:23Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"darwin/amd64"}

Install Docker (Windows user)

Install Chocolatey

Chocolatey installs in seconds. You are just a few steps from running choco right now!

First, ensure that you are using an administrative shell — you can also install as a non-admin, check out Non-Administrative Installation.
Copy the text specific to your command shell — cmd.exe or powershell.exe.
Paste the copied text into your shell and press Enter.
Wait a few seconds for the command to complete.
If you don’t see any errors, you are ready to use Chocolatey! Type choco or choco -? now, or see Getting Started for usage instructions.

Install Docker using Chocolateey

> choco install docker-desktop — pre

Step 1: React —

I used an npm package called create-react-app to quickly spin up a react application by running create-react-app sample-app

$ npx create-react-app my-app
npx: installed 98 in 6.337s

Creating a new React app in /Users/paulzhao/my-app.

Installing packages. This might take a couple of minutes.
Installing react, react-dom, and react-scripts with cra-template...

.
.
.Happy hacking!
$ cd my-app
$ npm start

What you’ll see

After this is done, you should see a folder structure like this:

Step 2: Docker —

If you’re at this point, we would need to create a docker image of this application and for this, we would be adding two files named below to the root of our application:

Dockerfile — Used for building the image that contains the optimized version of this application
Dockerfile.dev — Used for building the image that contains the development version which would be used to run tests

A Dockerfile file contains instructions that docker uses to build an image of an application

To keep things short, I’d just show you the actual code:

Dockerfile

#Split everything the build into two steps
FROM node:alpine as builder
# Above, we set the base image for this first stage as a light weigh node called alpine

WORKDIR '/app'
# Above we set the build environment as a folder called /app in the docker container to prevent clashes

COPY package.json .
# To prevent repeated npm installs anytime we make any change, we'd copy over the package.json and install things first

RUN npm i

COPY . .
# Copy the rest of the project over to the /app folder in the container

RUN npm run build
# Build the production version of our app in the container

FROM nginx 
# The image needs nginx to run on aws

EXPOSE 80 
#Nginx runs on port 80, so elastic beanstalk uses the expose command to expose this port

COPY --from=builder /app/build /usr/share/nginx/html

# Copy the content of the builder step, move the contents of build folder into the html folder in this nginx container
# That's where our app would run from in aws

# No need to specify a command to start nginx as it gets started by default when a container with the image starts

Dockerfile.dev

FROM node:alpine
# Above, we set the base image as a light weight node image called alpine

WORKDIR '/app'
# Above we set the build environment as a folder called /app in the docker container to prevent clashes

COPY package.json .
# To prevent repeated npm installs anytime we make any change, we'd copy over the package.json and install things first

RUN npm i

COPY . .
# Copy the rest of the project over to the /app folder in the container

CMD ["npm", "start"]
# Here we are setting the default command when a container is built and started up from this our image

At this point, you should create a new github repository (You can name it sample-docker-react like I did for consistency) and push the simple-app to it.

Your repository should look something like this:

Now we need to connect our repository to travis-ci and for this, we’d move on to:

Step 3: Travis-CI!

Authorising Travis CI to GitHub

Go to Travis CI and sign in with your GitHub. Just in case, you don’t have a Github account yet, you may create one here.

Authenticate your Travis account with GitHub

Logged in in Travis CI

Notes: Here you may encounter problems when logging in to your Travis CI after authentication on Firefox or Google Chrome. You resolve the problem, you need to login in different browser. In my case, Safari did the work. Then I simply copy logged in URLs for Travis CI and paste into my Firefox browser. Then I was allowed to login as shown above

This tells travis-ci to start monitoring the repository. but for this we need a YAML file called .travis.yml to be added to the application as this would contain the instructions on what you want travis-ci to do with your code when a build is triggered.

Here are some instructions you should add to your .travis.yml file:

sudo: required language: node_js
node_js:
    - "stable"
    
services: 
    - dockerbefore_install: 
    - docker build -t wilpat/sample-app -f Dockerfile.dev .script:     
    - docker run -e CI=true wilpat/sample-app npm run test -- --coverage

sudo: required- Travis ci requires the user to be with elevated permission
language: node_js- A way to specify the language that should be used in building the app
services- Used to tell travis-ci that we need an instance of docker running for this build
before_install- An array of things to do with your code before running tests or deploying your code(The instruction here builds the docker image using our Dockerfile.dev file and tags it with the name wilpat/sample-app)
scripts: An array of scripts to run before deploying your code.(Tests typically go here (The instruction here runs the our test by running image named wilpat/sample-app and overriding the command specified in the Dockerfile.dev)

We’d be needing one more instruction to tell travis-ci how and where to deploy our application, but we first need to setup AWS to receive our code.

Step 4: AWS Elastic Beanstalk —

Disclaimer: Nothing prepares you enough for the maze called AWS

AWS Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed with Java, . NET, PHP, Node. js, Python, Ruby, Go, and Docker on familiar servers such as Apache, Nginx, Passenger, and IIS.

We need to sign into AWS, hover on services and select Elastic Beanstalk then select Create New Application by the top right as seen below:

Provide application name as sample-docker-react and select Platform as Docker ; Platform branch as Docker running on 64bit Amazon Linux 2 ; Platform version as 3.1.0(recommended)

Click configure more options

Notes: I was unable to create the eb environment due to vpc issue. Keep note that VPC must be created with subnet and attached to a VGW. Otherwise, you may receive the two following errors

error 1 — for not creating VPC correctly in your chosen region. For me, it is us-east-1

Stack named ‘awseb-e-p752yct384-stack’ aborted operation. Current state: ‘CREATE_FAILED’ Reason: The following resource(s) failed to create: [AWSEBAutoScalingGroup].

error2 — for not attaching VGW to your chosen VPC

Service:AmazonEC2, Message:Network vpc-346c6f4e is not attached to any internet gateway

Below show how to modify network the vpc

Create EB successfully

Step 5: AWS IAM —

Creating a non-root user

Based on AWS best practice, root user is not recommended to perform everyday tasks, even the administrative ones. The root user, rather is used to to create your first IAM user, groups and roles. Then you need to securely lock away the root user credentials and use them to perform only a few account and service management tasks.

Notes: If you would like to learn more about why we should not use root user for operations and more about AWS account, please find more here.

**Keep credentials (Access key ID and Secret access key)**

Step 6: AWS S3 Buckets —

Amazon S3 or Amazon Simple Storage Service is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface.

Also, while creating an elastic beanstalk environment, AWS selects a server location closest to you for hosting it(You can select a different location too).

Also, after the environment is created, an S3 bucket is also created bearing a name that’s contains an identifier of this server location (e.g us-east-1, us-west-1) and it is used to store applications that you deployed in environments existing in this same server location.

Access to S3 bucket

Notes: When creating S3 bucket, keep in mind, S3 bucket name must be globally unique

Secondly, you need to specify a folder(aka bucket path in travis) that would be used by travis-ci for deployment

Goto Services > Storage > S3

Select the bucket with a name that has the identifier of the server location of your environment in it’s name.

Create a folder and you can name it the same name with your application( sample-docker-react `) this name choice is optional and I do it to help me remember what app it’s for.

Step 7: Back to Travis-CI

You’d need to save the AccessKeyID and SecretAccessKey as an environment variable in Travis-CI.

Go to your dashboard and select the sample-docker-react repository > click on the options by the top right > click on settings.

On this settings page you should add AWS_ACCESS_KEY and AWS_SECRET_KEY like I have done:

Step 8: Back to React —

We need to update our .travis-yml file with the instructions for deploying our code and then push to github.

Update yours with the following:

deploy:  provider: elasticbeanstalk  region: "us-east-1" ### AWS region of your choiceapp: "sample-docker-react" ### Your EB App nameenv: "SampleDockerReact-env" ### Your EB App environment namebucket_name: "elasticbeanstalk-us-east-2-XXXXXXXXXXXX" ### S3 bucket namebucket_path: "sample-app" ### S3 folder name under S3 bucket aboveon:
  branch: masteraccess_key_id: "$AWS_ACCESS_KEY"secret_access_key: "$AWS_SECRET_KEY"

deploy: Tells Travis-ci how to deploy your app
provider: Platform to be used
region: Where the server of your elastic beanstalk app environment exists(Update the file with yours)
app: Then name you gave your application
bucket_name: The s3 bucket generated for the server location used for your environment
bucket_path: The folder we created on aws to receive the app sent from travis-ci
on: Tells travis-ci which github branch update should trigger a build
access_key_id: IAM access key generated in AWS but stored in travis ci
secret_access_key: IAM secret key generated in IAM of AWS but stored in travis ci

Now our .travis.yml file should look something like:

sudo: required
services:
  - docker

before_install:
  - docker build -t lightninglife/sample-docker-react -f Dockerfile.dev .

script:
  - docker run -e CI=true lightninglife/sample-docker-react npm run test -- --coverage --watchAll=false

deploy:
  provider: elasticbeanstalk
  region: "us-east-1"
  app: "EBApptest"
  env: "Ebapptest-env"
  bucket_name: "s3-bucket-for-sample-docker-react"
  bucket_path: "EBApptest"
  on:
    branch: master
  access_key_id: "$AWS_ACCESS_KEY"
  secret_access_key: "$AWS_SECRET_KEY"

Step 9: Push to git —

Cross check your files and setup to ensure you did all I talked about in this walkthrough then Add, commit and push your updates to the repo on github.

Since we’ve added a .travis.yml file, travis-ci would pick up the update in few seconds and trigger a build.

After a successful build, Go to your application on AWS and you should see it get updated with the new code from travis, build the docker image, and start up the application which can be access when you click the link provided.

In Travis, sample-docker-react CI/ CD is built up automatically after git push from our local environment to Github

In AWS EB, click URL to get access to the application

Now anytime we update the master branch of our application, travis-ci picks up the changes and updates our application that lives in AWS.

Notes: Keep in mind few issues you may encounter while building the app through AWS EB

When choosing your docker as the platform, you may not choose the following. Otherwise, the docker image will throw your an error

Below are the error I found in eb logs

An error occurred during execution of command [app-deploy] - [Docker Specific Build Application].

Instead, choose the below option for your docker

Conclusion:

First of all, I’d like to say troubleshooting makes you a better and stronger DevOps Engineer. I learned this throughout this project.

It took me only a few hours to have everything else ready, but I spent days figuring out why Elastic Beanstalk was unable to deploy the app intended. I did research on tons of blogs and attempted a whole bunch of solutions, nothing turned out to be right. After running out solutions on my own, I reached out to my coach Broadus B. Palma Palmer (cloud/ engineer coach). I reattempted the solution suggested by him, then mysteriously resolved the issue. Experience matters! Never forget to reach out to your last resort — senior colleague or others when running out of solutions. And learning on its own is rewarding by all means!

Now let us recap what was being done in this project. In only 9 steps, we built up a fully functinal app via AWS EB using travis ci as CI/ CD tool. This project can be used to deploy any app of your choice on EB using travis ci, you may save it as a template.

In terms of Travis CI, I’d like to make a few comments. First of all, fully automatic deployment using Travis CI is pretty amazing. We only need to push our local environment to Github, .travis.yml will deploy your infrastructue in Travis CI. If it is successfully executed, the app will be deployed in AWS EB. Secondly, it’s just a reminder for Mac users — only Travis CI official website, it states the current version will not support Mac when using Docker.

Last but not least, keep throwing errors at me and keep learning and making progress on a daily basis!