Devops — CICD Pipeline 1: Deploying on AWS

9 min readAug 26, 2019

In this new series i want to show you how i deploy applications on AWS, creating my own CI/CD pipeline.

We will work to deploy our application using AWS Elastic Beanstalk, continuous integration doing Collaboration with AWS CodeCommit , continuous delivery using code Deploy, we will see different deployment strategies and we will also perform monitoring using Cloudwatch

Part 2 — CICD Pipeline — AWS Elastic Beanstalk

Part 3 — CICD Pipeline — Source control using git and CodeCommit

Part 4— CICD Pipeline — Unit Test with Code Build

What we will do in this Part 1:

Continuous Integration, We’re working on our code, we then decide to send our code to somewhere it can be built and packaged, then we carry out tests like unit test, at this stage we are satisfied with our code. now we are ready for continuous delivery.

CI = Code, Build and Test will all happen in our Local Environment

Continuous Delivery, we will first provision and environment to deploy our app, we will then deploy our app, then we will carry out test like acceptance test on the app. Acceptance test simply means clicking around the buttons or other elements of the app to make sure everything is working maybe say you need to show this to your superior.

CD = Provision, Deploy and Acceptance test, will happen in an environment we will provision for our application using Elastic Beanstalk

LETS DIG IN !!!

1. Create an IAM policy.

In this section, you will create an IAM customer-managed policy. Customer-managed policies provide more precise control over your policies than AWS managed policies. This policy will have permissions specific to the AWS resources you need for this tutorial

In the AWS Management Console, click Services, then click IAM to open the IAM dashboard.
In the left navigation menu, click Policies.
Click Create policy.
Click the JSON tab.
In the editor text-box, completely replace the sample policy with the following.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:*",
                "elasticbeanstalk:*",
                "ec2:*",
                "ecs:*",
                "ecr:*",
                "elasticloadbalancing:*",
                "autoscaling:*",
                "cloudwatch:*",
                "s3:*",
                "sns:*",
                "cloudformation:*",
                "dynamodb:*",
                "rds:*",
                "sqs:*",
                "logs:*",
                "events:*",
                "cloud9:*",
                "codecommit:*",
                "codebuild:*",
                "codepipeline:*",
                "codedeploy:*",
                "ssm:*",
                "es:*",
                "lambda:*",
                "tag:GetResources",
                "kms:ListKeyPolicies",
                "kms:GenerateRandom",
                "kms:ListRetirableGrants",
                "kms:GetKeyPolicy",
                "kms:ListResourceTags",
                "kms:ReEncryptFrom",
                "kms:ListGrants",
                "kms:GetParametersForImport",
                "kms:ListKeys",
                "kms:GetKeyRotationStatus",
                "kms:ListAliases",
                "kms:ReEncryptTo",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        }
    ]
}

Click Review Policy.
For Name, type myDeployingPolicy (or give it any name you want)
Click Create policy.
You have successfully created an IAM policy. When you create IAM policies, follow the Golden security advice of granting least privilege — that is, granting only the permissions required to perform a task. Determine what users need to do and then craft policies for them that let the users perform only those tasks.

2. Create an IAM user and attach a policy to the user.

In this section, we will create an IAM user and attach the policy to the user. If you are familiar with IAM users, you may want to attempt to complete this section before reading the step-by-step instructions.

IAM user name: myDeployingUser
Access type: AWS Console access
Policy: myDeployingPolicy
Important Make a note of the password for the myDeployingUser and the sign-in URL for the IAM user. You should see the sign-in URL in the success message at the top.

Follow the steps below to create an IAM user

In the AWS Management Console, click Services, then click IAM to go to the IAM dashboard.
In the left navigation menu, click Users.
Click Add user.
In the User name text box, type myDeployingUser
For Access type, select AWS Console access.
For Console password, you may choose either Autogenerated password or Custom password. If you choose Autogenerated, you will be prompted to change your console password when you log in to the AWS Management Console as the myDeployingUser user. Make a note of the password.
Click Next: Permissions.
Under Set permissions for myDeployingUser section, click Attach existing policies directly.
In the search text box for Filter, type myDeployingPolicy Select myDeployingPolicy from the filtered list.
Click Next: Review.
Review the information and click Create user. You should see a success message.
Make sure to note the password for the myDeployingUser.
Note the sign-in URL in the success message at the top. This is a special URL for IAM users, which includes your account ID.
Sign out of the console, and sign back in as the myDeployingUser IAM user.

3. Create an AWS Cloud9 environment.

In this section, you will create an AWS Cloud9 environment and explore the environment. (This can also be done with an IDE of your choice on your local pc)

AWS Cloud9 environment name: DeployingOnAWS
Network and Instance type settings: Keep the default settings.

Sign in to the AWS Management Console as the myDeployingUser IAM user.
In the console, click Services, then click Cloud9 to open the Cloud9 dashboard.
Make sure you are in the Oregon region.
Click Create environment at the top-right corner.
For Name, type DeployingOnAWS
Click Next step.
On the Configure Settings page, leave the default settings and click Next step.
Review the details and click Create environment. This should launch your AWS Cloud9 environment in a few minutes.
Upon environment creation, notice the terminal window on the bottom pane. The terminal provides a remote login to the instance on which the AWS Cloud9 environment is hosted, just as you use SSH for remote login . A pre-authenticated AWS CLI is installed in your terminal.

4. Download the application code and set up the local AWS Cloud9 environment.

Make sure you are in the home directory of your AWS Cloud9 environment by running the command below in the AWS Cloud9 terminal.

cd ~/environment

To download the application code, run the command below in your AWS Cloud9 terminal.

wget https://us-west-2-tcdev.s3.amazonaws.com/courses/AWS-100-ADD/v1.0.0/exercises/ex-cloud9.zip -O ~/ex-cloud9.zip

Unzip the application code by running the command below.

unzip -o ~/ex-cloud9.zip

The FlaskApp/requirements.txt file has the list of requirements needed to be installed for the application to run. Install the requirements by running the command below.

sudo pip-3.6 install -r FlaskApp/requirements.txt

Start the MySQL service by running the command below.

sudo service mysqld start

Run the command below to make sure the MySQL service starts every time the AWS Cloud9 environment hibernates and comes back online.

sudo chkconfig mysqld on

Once you start the MySQL service, you will see a message as shown in the screenshot below to set a password for the MySQL root user.

To set the MySQL root user, run the command below. Make sure to replace REPLACE_WITH_ROOT_PASSWORD with the Local / Cloud9 root password on your password sheet.

mysqladmin -u root password REPLACE_WITH_ROOT_PASSWORD

Open the SetupScripts/create_schema.sql file. Locate the variable REPLACE_WITH_WEB_USER_PASSWORD in the file and replace it with the Local / Cloud9 web_user_password on your password sheet.
Save the SetupScripts/create_schema.sql file.
Change the working directory to SetupScripts folder by running the command below.

cd SetupScripts/

To create the database schema, run the create_schema.sql script. Type the command below in your AWS Cloud9 terminal.

mysql -h localhost -u root -p < create_schema.sql

You should see a prompt to enter the root password. Enter the Local / Cloud9 root password on your password sheet.
To populate the database with the application data, you will run the SetupScripts/database_populate.py Python file. Run the command below. Make sure to replace REPLACE_WITH_ROOT_PASSWORD with the Local / Cloud9 root password on your password sheet.

PASSWORD=REPLACE_WITH_ROOT_PASSWORD python3 database_populate.py

5. Run the application code.

To run the application code, open the FlaskApp/application.py file.
On the top menu bar, click Run -> Run With -> Python 3.
A run configuration window pane should open up at the bottom. Save the Python 3 run configuration for future runs of the application code. Type Python3Config in the text-box next to the Run button as shown in the screenshot below.

When you run the application.py file, you should see an error message like the one below.

KeyError: ‘DATABASE_HOST’

To set the environment variables, click ENV at the right side in the run configuration pane at the bottom. Refer to the screenshot below.

Fill the values in the environment variables list against Name and Value as shown in the table below.

DATABASE_HOST => localhost
DATABASE_USER => web_user
DATABASE_PASSWORD => Type the Local / Cloud9 web_user password on your password sheet.
DATABASE_DB_NAME => routes

Click the Run button at the left side in the run configuration pane at the bottom.
You should see a message similar to the one below.

Running on http://0.0.0.0:8080/

6. Test the application.

To test the application, click Preview -> Preview Running Application on the top menu bar of the AWS Cloud9 environment. Refer to the screenshot below.

You should see the application running in a small window in the AWS Cloud9 environment.
Pop out the application in a new window by clicking the Pop Out button shown in the screenshot below.

The application is now running in your local AWS Cloud9 environment.
Feel free to play around with the application to choose the shortest distance. You will need to choose one image which could be the shortest distance between the airports listed under the images. Choosing the correct image will score you a point. There are three levels — easy, medium and hard — and each category has three images to play around with.

Your application is now running on port 8080.

7. Run static code analysis and unit tests on the application code.

In your AWS Cloud9 terminal window, change your working directory to the FlaskApp folder by running the command below.

cd ~/environment/FlaskApp/

To run the static code analysis and the unit tests for the application code, run the FlaskApp/local_build.sh script by executing the command below.

./local_build.sh

You should see a message that the code has been rated 10.00/10 and all the units tests have run and passed. Refer to the screenshot below.

To explore the FlaskApp/local_build.sh script, open the FlaskApp/local_build.sh script file.
Notice that pylint is used to do a static code analysis on the application code. Nosetests runs a set of unit tests against the application code. Open the FlaskApp/test_application.py to have a look at the unit tests that are run against the application code. You will notice that there are four unit tests — test_home, test_get_challenge , test_get_route_miles, test_get_route_miles_bad_route — that test the functionality of the application. The database logic has been mocked to test the application logic in isolation.
As seen in the screenshot above, the unit tests are covering a 100% of the code. This code coverage information is populated in the FlaskApp/cover folder.
Select the FlaskApp/cover/index.html in the left side tree view, right-click and click Preview. You should see the code coverage report.

Congratulations! You successfully set up your local environment with a running application, did a static code analysis, and unit-tested the application.

Summary

In this part 1 we have learnt how to

Create I Am Policy
Create a new user and assign that policy to the user taking note of the Golden security advice of granting least privilege
and then we used aws Cloud9 to create a local environment for our app

We will be moving on to the part 2 of this series where we will handle using AWS Beanstalk for our deployment.

Follow me to get notified for the next parts

credit goes to eDX.org,please visit this amazing website for more knowledge on modern technologies