Complete AWS CI/CD Pipeline
DevOps Project-Project 3
Project Source: DevOps Project by Imran Teli
GitHub repo: Complete AWS CI/CD Project
AWS Services Used:
- AWS Code Commit
- AWS Code Build
- AWS Code Deploy
- AWS Beanstalk
- AWS Simple Store Service (S3)
- Amazon Relational Database System (RDS)
Project Architecture
Flow of Execution
- Login to AWS account
- Create key pair for Beanstalk instance login
- Create Elastic Beanstalk Environment and Applications
- RDS and App Setup on Beanstalk
- Validating Elastic Beanstalk Environment and Applications Created
- Build Artifact locally and Manual Deploy Artifact
- Configure the application environment load balancer
- Upload Artifact to Elastic Beanstalk
- Create CodeCommit Repository
- Code Build with build spec file
- Create Pipeline and Deploy
- Validate that the application deployed successfully
- Test the endpoint URL and the login page
Step 1: Create Key pair for Beanstalk EC2 Login
Create a key pair that will be used to log in to Elastic Beanstalk. Navigate to EC2 on AWS console, select key pair, and click create key pair.
Note:
vprofile-bean-key
Ensure to note the location on your machine where the private key is downloaded as it will be needed to log in via SSH into the instance.
Step 2: Create Beanstalk Environment and Applications
Note:
Name: vprofile-App
Platform: Tomcat
keep default for platform branch and platform version
Application code: sample application
Click Configure more options:
select Custom configuration
Edit Instances
Root Volume (Boot Device): General Purpose(SSD)
Size: 8GB
EC2 SecGrp: vprofile-backend-SG
Keep others as default and save
Edit Capacity
LoadBalanced
Min:2
Max:4
InstanceType: t2.micro
metric: networkout
Keep others as default and save
Edit Load Balancer
Load balancer Type: Application Load Balancer
Percentage :50 %
Keep others as default and save
Edit Rolling updates and deployments
Deployment policy: Rolling
Percentage :50 %
Keep others as default and save
Edit Security
EC2 key pair: vprofile-bean-key
Step 3: Create Database and App Setup on Beanstalk
Note:
Method: Standard Create
Engine Options: MySQL
Engine version: 5.7.34
Templates: Free-Tier
DB Instance Identifier: vprofile-bean-rds
Master username: admin
Password: Auto generate psw
Instance Type: db.t2.micro
No public access
Subnet grp: vprofile-bean-rds-sg
SecGrp: vprofile-bean-rds-sg
Database port: 3306
DB Authentication: Password authentication
Additional Configuration
Initial DB Name: accounts
Keep or add additional configuration according to your preference
- Validate Elastic Beanstalk Environment and Applications Created
Check the Beanstalk environment and Ec2 instances created by the Beanstalk
Update the Security Group on the instance to only allow traffic from your IP and not from anywhere (0.0.0.0/0)
Update the RDS Security Group to allow the Beanstalk instance to connect RDS on 3306
SSH into the Beanstalk instance to deploy the Schema files to the Beanstalk instance
sudo -i,
yum isntall git mysql -y
Clone the source code from repo https://github.com/Ikenna1o2/vprofile-project.git
cd vprofile-project
git checkout vp-rem
check the properties file in the src directory: ls src/main/resources/db_backup.sql
Login into the database:
mysql -h vprofile-bean-rds.czsy3j7gysd6.us-east-1.rds.amazonaws.com -u admin -pS0aBROo5fbKJPL6qEJYY accounts
Deploy the resources/db_backup to the database:
mysql -h vprofile-bean-rds.czsy3j7gysd6.us-east-1.rds.amazonaws.com -u admin -pS0aBROo5fbKJPL6qEJYY accounts < src/main/resources/db_backup.sql
Step 4: Build Artifact locally and Manual Deploy Artifact
Clone the source code from repo https://github.com/Ikenna1o2/vprofile-project.git to your local system
cd vprofile-project
git checkout vp-rem
Update the application.properties file with the correct endpoints and username and password.
This is found in the src/main/resources directory of the source code cloned to your local system.
vim src/main/resources/application.properties
cd back to the root directory of the project where the pom.xml file is located.
Run mvn install command to build the artifact.
Copy and move to your desktop home directory for easy upload: cp target/vprofile-v2.war ~/Desktop/
- Next, we need to configure the application environment load balancer.
To do this, navigate to the Elastic Beanstalk service in the AWS console, under the app environment, click Configuration and make changes to the Listener and processes section, and apply them.
Add to Listener HTTPS port 443 with SSL cert
Processes: Health check path : /login
Stickiness policy: enabled
- Upload Artifact to Elastic Beanstalk
Navigate back to the AWS console Elastic Beanstalk, click on the Application versions, and select the Upload button on the top right corner to upload the artifact from your local.
It will automatically upload the artifact to an S3 bucket created by Elastic Beanstalk.
Select the uploaded application and click Deploy.
Click on the endpoint to see the login page. and login integrated
- Next, is to deliver our artifact automatically to Beanstalk
Instead of using GitHub for the version control, we will be using AWS Code Commit which is AWS managed version control service
Step 5: Create CodeCommit Repository
Navigate back to the AWS console, Search for CodeCommit, Click on create Repository
In this project, we will be using SSH to connect to our repository which is the preferred method.
Also, we need to create an IAM user and also create a policy.
On IAM service, click create user
Click on create policy to create policy to allow only access to the repo we created.
choose a service
Search for and select CodeCommit
Resources: specific
repository: add ARN
Region: us-east-1
Repository name: vprofile-code-repo
Give the policy a name
Go to IAM, search for the newly created policy on the attached policy and select the policy and create a user
Generate SSH key from the local system and upload to IAM user
ssh-keygen
name the key: /c/Users/ikenn/.ssh/vpro-codecommit_rsa
cd ~/.ssh
cat vpro-codecommit_rsa.pub
Copy the public key and upload it to the IAM user
- Next, Create a config file using the SSH Key ID on the user
Host git-codecommit.*.amazonaws.com
User APKATC5XPJAIKUILORPO
IdentityFile ~/.ssh/vpro-codecommit_rsa
Change the permission of the config file to 600;
chmod 600 config
Authentication over SSH with AWS CodeCommiit
ssh git-codecommit.us-east-2.amazonaws.com
Switch to your repository already cloned on your local system
git branch -a (list all branches)
git checkout master
git branch -a | grep -v HEAD | cut -d ‘/’ -f3 | grep -v master
git branch -a | grep -v HEAD | cut -d ‘/’ -f3 | grep -v master > /tmp/branches
for i in `cat /tmp/branches`; do git checkout $i;done
git remote rm origin (remove remote origin)
- Next, Click on Clone URL on AWS CodeCommit to clone SSH URL
git remote add origin ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/vprofile-code-repo (add the AWS CodeCommiit SSH URL)
cat .git/config (validate clone CodeCommit repo)
git push origin — all (push repo to AWS CodeCommit)
Validate all branches are on the AWS CodeCommit repository
Step 6: Code Build with build spec file
Note:
Click on the CodeBuild service on the CodeCommiit
Click Create Project and name your name
Source Provider: Repository: select our created repository
Branch: vp-rem
Environment: Managed Image
Operating System: Ubuntu
Runtime: Standard
Image: aws/codebuild/standard:4.0
Environment Type: linux
Build Spec: insert build commands
Swtich to editor
copy and paste the build spec file and update the user, password and rds endpoint with the new information.
Below, is the Build Spec Yaml File used, which is also in the GitHub repo for this project;
Artifact: Type: Amazon S3
bucket name: select an existing bucket on your S3
Logs: Groupname: vprofile-cicd-project
Stream Name: buildlogs
Go back to CodeBuild. Scroll down and click on “Create build project”.
The project will start building
Step 7: Create Pipeline and Deploy
Note:
Click Pipeline
create pipeline and name your pipeline
source: Source provider: AWS CodeCommiit
Repository name: vprofile-code-repo
Branch name: vp-rem
Build provider: AWS CodeBuild
Region: Us East (N. virginia)
Project name: vprofile-Build
Depoly provider: AWS Elastic Beanstalk
Application name: vprofile-App
Environment name: vprofileapp-env
click create pipeline
The pipeline is created and the job was triggered immediately
Pipeline was successful and deployed.
Validate the app on the elastic Beanstalk and Verify that the application deployed successfully.
Step 8: Test the endpoint URL and the login page
Login page
App is operational
Finally Clean-up
Ensure to delete all resources created throughout the project to avoid charges.
Thank you for reading. please react and send in your comments and don’t forget to connect with me on LinkedIn HERE .
See you on the next project, Keep learning and practicing.
