DevSecOps: Deploying a YouTube Clone App on AWS EKS with Jenkins and Terraform
Introduction:
In today’s fast-paced development landscape, DevSecOps practices have become crucial for efficiently delivering secure and scalable applications. Automating the deployment process while integrating security measures from the outset ensures that applications are not only deployed quickly but also adhere to stringent security standards.
In this blog post, we’ll explore how to leverage the power of Jenkins and Terraform to deploy a YouTube clone application on Amazon Elastic Kubernetes Service (EKS) securely. By following this guide, you’ll learn how to orchestrate the deployment process, manage infrastructure as code, and integrate security practices seamlessly into your development pipeline.
We’ll start by setting up a Kubernetes cluster on AWS EKS using Terraform, enabling us to define our infrastructure requirements in code. Next, we’ll configure Jenkins, a popular continuous integration and continuous delivery (CI/CD) tool, to automate the deployment process. Leveraging Jenkins pipelines, we’ll define the steps to build, test, and deploy our YouTube clone application efficiently.
Security is a top priority in any deployment process, especially when dealing with sensitive data or user information. Throughout this guide, we’ll incorporate security best practices, such as image scanning, secrets management to ensure that our deployment remains secure at every stage.
By the end of this tutorial, you’ll have a comprehensive understanding of how to implement a robust DevSecOps workflow, empowering you to deploy applications confidently while maintaining a high level of security and compliance.
Let’s dive in and discover how to deploy your YouTube clone app on AWS EKS with Jenkins and Terraform.
STEPS:
Step:1 :- Creation of Jenkins Server
GitHub Repo: https://github.com/uniquesreedhar/Youtube-clone-app.git
- Create an user with any name on AWS console and store the access keys.
- Install & Configure Terraform and AWS CLI on your local machine to create Jenkins Server on AWS Cloud.
#Terraform Installation Script
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg - dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update
sudo apt install terraform -y
#AWSCLI Installation Script
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip -y
unzip awscliv2.zip
sudo ./aws/install3. Run the below command, and add your keys.
aws configure4. Clone the GitHub Repository.
git clone https://github.com/uniquesreedhar/Youtube-clone-app.git5. Navigate to the Jenkins-Server-TF
6. Do some modifications to the backend.tf file such as changing the bucket name and dynamodb table(make sure you have created both manually on AWS Cloud).
7. Now, you have to replace the Pem File name as you have some other name for your Pem file with the one that is already created on AWS.
8. Initialize the backend by running the below command.
terraform init9. Run the below command to get the blueprint of what kind of AWS services will be created.
terraform plan -var-file=variables.tfvars10. Now, run the below command to create the infrastructure on AWS cloud which will take 3 to 4 minutes maximum.
terraform apply -var-file=variables.tfvars --auto-approve
11. Now, connect to your Jenkins-Server by clicking on Connect.
12. Copy the ssh command and paste it on your local machine .
Step:2 :- Configure Jenkins Server.
- Copy the public IP of your Jenkins Server and paste it on your favorite browser with an 8080 port.
2. Now, run the below command to get the administrator password and paste it on your Jenkins.
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
3. Create user in case if you want to give own password every time it get restarted.
4. Navigate to manage Jenkins → Plugins → Available Plugins.
Install the following Plugins:
1 → Eclipse Temurin Installer (Install without restart)
2 → SonarQube Scanner (Install without restart)
3 → NodeJs Plugin (Install Without restart)
4 → Docker
5 → Docker commons
6 → Docker pipeline
7 → Docker API
8 → Docker Build step
9 → Owasp Dependency Check
10 → Terraform
11 → Kubernetes
12 → Kubernetes CLI
13 → Kubernetes Client API
14 → Kubernetes Pipeline DevOps steps
15 → AWS Credentials
16 → Pipeline: AWS Steps
17 → Slack notifications
18 → Splunk
5. Grab the Public IP Address of your EC2 Instance, SonarQube works on Port 9000, so <Public IP>:9000, go to your SonarQube server.
Username: admin
password: admin
6. Update the password.
7. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and Click on Generate Token.
8. Copy Token
9. In the SonarQube Dashboard add a quality gate also,
Administration–> Configuration–>Webhooks
10. Click on Create
Name: Jenkins
#in url section of quality gate
<http://jenkins-public-ip:8080>/sonarqube-webhook/
11. We had to configure the Tools.
jdk17:
sonar-scanner:
nodejs:
DP-Check:
Docker:
12. Go to Jenkins Dashboard → Manage Jenkins → Credentials
Add these credentials:
AWS:
GitHub :
Sonar-token:
Docker:
13. Now, go to Dashboard → Manage Jenkins → System and Add sonar server.
Step:3 :-Launch and Configure Splunk Server.
- Navigate to the EC2 service and launch Ubuntu 22.04 instances. Ensure you select T2.medium as the instance type and allocate 24GB of storage to each instance.
2. Connect to that instance using the SSH with your pem file.
3. Install Splunk on your Ubuntu instance:
wget -O splunk-9.1.1-64e843ea36b1-linux-2.6-amd64.deb "https://download.splunk.com/products/splunk/releases/9.1.1/linux/splunk-9.1.1-64e843ea36b1-linux-2.6-amd64.deb"
sudo dpkg -i splunk-9.1.1-64e843ea36b1-linux-2.6-amd64.deb
4. Please note that after running this command, you should follow the on-screen prompts to accept the terms and complete the setup to 100%.
sudo /opt/splunk/bin/splunk enable boot-startAfter completing the initial setup and accepting the terms, you’ll be prompted to create an admin user.
Administrator Username: Choose a username for the admin account. This should be a unique and secure username.
Administrator Password: Set a strong and secure password for the admin account.
5. Run the following commands one after the other.
sudo ufw allow openSSH
#By running this command, you ensure that SSH access is permitted through your firewall, which is crucial for remote server management and administration.
sudo ufw allow 8000
# It permits access to a specific port for network services or applications.
sudo ufw status
sudo ufw enable
#By running these commands, you can both check the current status of your firewall and activate it to apply the defined rules and settings.
6. Run this command for allowing you to begin using the Splunk platform for data analysis, monitoring, and other data-related tasks.
sudo /opt/splunk/bin/splunk start
7. Configure security group of Splunk server for port 8000.
8. Access the application on <public_ip of splunk server>:8000
Login with your credentials:
9. Navigate to Apps → Find more apps
10. Search for Jenkins and click on install.
11. To Login and install you need an Splunk account create one by following the video: https://www.youtube.com/watch?v=njTY4tf7ajg
by Ajay Kumar Yegireddi sir.
12. Click on Login and install.
13. In the home Navigate to Administrator → Data Inputs (Under DATA section)
14. Click on “HTTP Event Collector”.
15. Go to Global settings and add configuration as below.
16. Click on “New Token”
17. Provide a name for it and click on next.
18. Review and submit it.
19. Copy generated token.
20. Navigate to Dashboard →Manage Jenkins → System
Give that token and Splunk server IP as Host.
21. In Splunk machine run this command:
sudo ufw allow 808822. Now in the System under Splunk click on Test connection
Step:4 :- Integrate Slack for Notifications
- Create a slack account by following the instructions in a video by Ajay Kumar Yegireddi sir : https://www.youtube.com/watch?v=9ZUy3oHNgh8
- Go to Slack and click on the name → Tools and Settings → Manage apps
3. Then in a new tab search for Jenkins CI and click on it.
4. Click on Add to Slack
5. Now choose your Slack channel and click on Add Jenkins CI integration.
6. Copy the team subdomain and integration token credential ID for later use.
7. Add this token under credentials of jenkins with ID “slack”
8. Configure the token under slack in Manage Jenkins → System.
Workspace : Subdomain
Default channel: #jenkins
Then test connection:
9. This will create an integration on slack UI:
Step:5 :- Create an API key from Rapid API
- Open a new tab in the browser and search for rapidapi.com
- It will automatically provide your mail and select a mail to create an account.
- Account is created.
4. Now in the search bar search for YouTube and select YouTube v3.
5. Copy API and use it in the Jenkins file under docker build and push section.
Step:6 :- Create Jenkins CI pipeline
- Up to this lets create a pipeline and check if there gone anything wrong.
- Click on “New Item”
- Provide a name for it.
4. Under Pipeline section select “Pipeline script from SCM”
Then our GitHub repo URL.
Then configured GitHub credentials.
5. Click on “Apply” and then build.
Upon successful completion console will be like:
Slack UI:
Splunk UI:
Dependency Checker:
SonarQube Analysis:
Image will be pushed to Docker Hub:
6. You will get an email of status of Jenkins pipeline as:
7. Access application deployed as docker container on Jenkins server by pasting <ip>:3000 on browser.
Step:7:- Create Jenkins pipeline for EKS
- Click on “New Item” and provide a name for it and click on “OK”.
2. Under pipeline section give “Pipeline script from SCM ” and then git then provide credentials.
3. Give branch as “main” and give path for Jenkinsfile.
4. Click on apply and then build it.
Upon successful creation console look as:
5. This will create a cluster on AWS with node groups.
Step:8:- Configure Jenkins pipeline for Deployment to EKS
- Run the below command to configure the EKS Cluster on Jenkins server.
aws eks update-kubeconfig --region <region> --name <cluster_name>
#Example
aws eks update-kubeconfig --region us-east-1 --name YouTube-EKS-Cluster
#Then run
kubectl get pods2. This will give an error stating “asked to provide credentials”.
3. To solve this we need to install aws-iam-authentiactor.
4. Download the Amazon EKS-vended aws-iam-authenticator binary from Amazon S3:
curl -o aws-iam-authenticator https://amazon-eks.s3.us-west-2.amazonaws.com/1.15.10/2020-02-22/bin/linux/amd64/aws-iam-authenticator5. Apply execute permissions to the binary:
chmod +x ./aws-iam-authenticator6. And move it into a common directory:
sudo mv ./aws-iam-authenticator /usr/local/bin7. Configure the keys also:
aws configure
8. Copy the config file and save it as secret.txt.
9. In Jenkins Under credentials add the file as “secret file” with id “k8s”.
10. Add this stage to the existing pipeline.
stage('Deploy to kubernets'){
steps{
withAWS(credentials: 'aws-key', region: 'us-east-1') {
script{
withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
sh 'kubectl apply -f deployment.yml'
}
}
}
}
}11. Then build the pipeline once Again:
On successful exicution:
The whole Jenkins file is:
def COLOR_MAP = [
'FAILURE' : 'danger',
'SUCCESS' : 'good'
]
pipeline{
agent any
tools{
jdk 'jdk17'
nodejs 'node16'
}
environment {
SCANNER_HOME=tool 'sonar-scanner'
}
stages {
stage('clean workspace'){
steps{
cleanWs()
}
}
stage('Checkout from Git'){
steps{
git branch: 'main', url: 'https://github.com/uniquesreedhar/Youtube-clone-app.git'
}
}
stage("Sonarqube Analysis "){
steps{
withSonarQubeEnv('sonar-server') {
sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=youtube \
-Dsonar.projectKey=youtube '''
}
}
}
stage("quality gate"){
steps {
script {
waitForQualityGate abortPipeline: false, credentialsId: 'sonar-token'
}
}
}
stage('Install Dependencies') {
steps {
sh "npm install"
}
}
stage('OWASP FS SCAN') {
steps {
dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
}
}
stage('TRIVY FS SCAN') {
steps {
sh "trivy fs . > trivyfs.txt"
}
}
stage("Docker Build & Push"){
steps{
script{
withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){
sh "docker build --build-arg REACT_APP_RAPID_API_KEY='e9fa205003msh376a022364' -t youtube ."
sh "docker tag youtube sreedhar8897/youtube:latest "
sh "docker push sreedhar8897/youtube:latest "
}
}
}
}
stage("TRIVY"){
steps{
sh "trivy image sreedhar8897/youtube:latest > trivyimage.txt"
}
}
stage('Deploy to container'){
steps{
sh 'docker run -d --name youtube1 -p 3000:3000 sreedhar8897/youtube:latest'
}
}
stage('Deploy to kubernets'){
steps{
withAWS(credentials: 'aws-key', region: 'us-east-1') {
script{
withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
sh 'kubectl apply -f deployment.yml'
}
}
} }
}
}
post {
always {
echo 'Slack Notifications'
slackSend (
channel: '#jenkins',
color: COLOR_MAP[currentBuild.currentResult],
message: "*${currentBuild.currentResult}:* Job ${env.JOB_NAME} \n build ${env.BUILD_NUMBER} \n More info at: ${env.BUILD_URL}"
)
}
}
}Under Splunk you can check:
Audit Trails:
Jenkins Health:
Build Analysis:
Alerts:
Now check this created the resources on EKS by running:
kubectl get all
This will create a load balancer on AWS:
To enable access for load balancer to access Node Open Port 80 on node security group.
Copy the DNS name and paste it on your browser:
Test it by playing a video under different categories:
Step:9:- Clean Up of Resources
- In Jenkins of EKS -Terraform pipeline click on “Build with parameters” and action as “destroy”.
2. Upon successful execution this will delete EKS cluster.
3. Then terminate the Splunk Server:
4. In local in “Jenkins-Server-TF” run this command to destroy the jenkins server:
terraform destroy --var-file=variables.tfvars --auto-approve
Thank you for reading my blog… :)
If you find this worth reading please follow and clap for me ..:)
