Real-Time DevSecOps Pipeline for a DotNet Web App

Steps:-

Step 1 — Create an Ubuntu T2 Large Instance with 30GB storage

Step 2 — Install Jenkins, Docker and Trivy. Create a Sonarqube Container using Docker.

Step 3 — Install Plugins like JDK, Sonarqube Scanner

Step 4 — Install OWASP Dependency Check Plugins

Step 5— Configure Sonar Server in Manage Jenkins

Step 6 — Create a Pipeline Project in Jenkins using Declarative Pipeline

Step 7 — Install make package

Step 8 — Docker Image Build and Push

Step 9 — Deploy the image using Docker

Step 10 — Access the Real World Application

Step 11 — Kubernetes Set Up

Step 12 — Terminate the AWS EC2 Instance

Step 1 — Launch an AWS T2 Large Instance. Use the image as Ubuntu. You can create a new key pair or use an existing one. Enable HTTP and HTTPS settings in the Security Group.

let’s connect to your ec2 via ssh using command ssh -i “laptopkey.pem” ubuntu@ec2–65–0–92–82.ap-south-

1.compute.amazonaws.com

run the following commands

sudo su
apt update
clone the github repo by
git clone https://github.com/zeeshancli/Dotnet

Step 2 — Install Jenkins, Docker and Trivy. Create a Sonarqube Container using Docker.

2A — To Install Java and Jenkins

Connect to your console, and enter these commands to Install Java,latest version and Jenkins

sudo apt update
sudo apt install openjdk-17-jdk 
sudo apt install openjdk-17-jre

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \ https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \https://pkg.jenkins.io/debian-stable binary/ | sudo tee \/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins
sudo systemctl enable jenkins 
sudo systemctl start jenkins 
sudo systemctl status Jenkins

Once Jenkins is installed, you will need to go to your AWS EC2 Security Group and open Inbound Port 8080, since Jenkins works on Port 8080

Now, grab your Public IP Address

<EC2 Public IP Address:8080>

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Unlock Jenkins using an administrative password and install the required plugins.

Jenkins will now get installed and install all the libraries.

2B — Install Docker

apt-get install docker.io -y
usermod -aG docker $USER # Replace with your username e.g 'ubuntu'
newgrp docker
sudo chmod 777 /var/run/docker.sock

After the docker installation, we create a sonarqube container (Remember added 9000 port in the security group)

run the following commands to install and run the conatainer of sonarqube on port no. 9000

docker run -itd - name sonar -p 9000:9000 sonarqube:lts- community

Now browse http://your_public_ip:9000 you will

see sonarqube is running

username =admin password=admin

2C — Install Trivy

sudo apt-get install wget apt-transport-https gnupg lsb-release
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update 
sudo apt-get install trivy

now your trivy is ready to check and scan your image for any vulnerabilities

to check run the following commands

trivy image <image id>

Step 3 — Install some suggested plugins for pipeline to run without errors

Goto Manage Jenkins →Plugins → Available Plugins → Install below plugins

1 → Eclipse Temurin Installer (Install without restart)

2 → SonarQube Scanner (Install without restart)

3 → OWASP Dependency Check (Install without restart)

3A — Configure Java in Global Tool Configuration

Goto Manage Jenkins → Tools → Install JDK17→ Click on Apply and Save

1. click on add jdk and select installer adoptium.net

2. choose jdk-17.0.10+7 version and in name section enter jdk 17

Next is to add dependency check

1. add dependency check
2. name = DP-Check
3. from add installer select install from github.com

Click on apply and Save here.

Grab the Public IP Address of your EC2 Instance, Sonarqube works on Port 9000, <Public IP>:9000. Goto your Sonarqube Server. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and click on Generate Token

Click on Update Token

Create a token with a name and generate

Copy this Token

Goto Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this

Enter the Token in secret field

You will this page once you click on create

Step 5 — Configure Sonar Server in Manage Jenkins

Now, go to Dashboard → Manage Jenkins → Configure System→ add sonarqube servers

1. name =sonar-server
2. server_url=http://public_ip:9000
3. server authentication token = sonar-token

Click on Apply and Save

The Configure System option is used in Jenkins to configure different server

Global Tool Configuration is used to configure different tools that we install using Plugins

We will install a sonar scanner in the tools.

Go To Dashboard → Manage Jenkins → Tools

1. add sonar scanner
2. name =sonar-scanner

In the Sonarqube Dashboard add a quality gate also

Administration → Configuration →Webhooks

Click on create

#in url section of quality gate <http://jenkins-public-ip:8080>/sonarqube-webhook/

Click on Apply and Save here.

Step 6 — Create a Pipeline Project in Jenkins using Declarative Pipeline

1. go to new item →select pipeline →in the name section type dotnet

2. scroll down to the pipeline script and copy paste the following code

pipeline {
    agent any
    tools {
        jdk 'jdk17'
    }
    environment {
        SCANNER_HOME = tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace') {
            steps {
                cleanWs()
            }
        }
        stage('Checkout From Git') {
            steps {
                git branch: 'main', url: 'https://github.com/zeeshancli/Dotnet'
            }
        }
        stage("Sonarqube Analysis ") {
            steps {
                withSonarQubeEnv('sonar-server') {
                    sh "$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Dotnet-Webapp -Dsonar.projectKey=Dotnet-Webapp"
                }
            }
        }
        stage("quality gate") {
            steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
                }
            }
        }
        stage("TRIVY File scan") {
            steps {
                sh "trivy fs . > trivy-fs_report.txt"
            }
        }
        stage("OWASP Dependency Check") {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --format XML', odcInstallation: 'DP-Check'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
    }
}

Click on Build now, you will see the stage view like this

To see the report, you can go to Sonarqube Server and go to Projects.

Step 7 — Install make package

sudo apt install make
 # to check version install or not make -v

Step 8 — Docker Image Build and Push

We need to install the Docker tool in our system, Goto Dashboard → Manage Plugins → Available plugins → Search for Docker and

install these plugins

•  Docker
•  Docker API 
•  Docker Commons
•  Docker API 
• docker-build-step

and click on install without restart

Now, goto Dashboard → Manage Jenkins → Tools

1. click on add docker
2. name=docker
3. add installer =download from docker.com

Click on apply and save

Now go to Dashboard → Manage Jenkins → Credentials
Add DockerHub Username and Password under Global Credentials

In the makefile, we already defined some conditions to build, tag and push images to dockerhub.

that’s why we are using make image and make a push in the place of

docker build -t and docker push

Add this stage which is highlighted in bold to Pipeline Script

pipeline {
    agent any
    tools {
        jdk 'jdk17'
    }
    environment {
        SCANNER_HOME = tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace') {
            steps {
                cleanWs()
            }
        }
        stage('Checkout From Git') {
            steps {
                git branch: 'main', url: 'https://github.com/zeeshancli/Dotnet'
            }
        }
        stage("Sonarqube Analysis ") {
            steps {
                withSonarQubeEnv('sonar-server') {
                    sh "$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Dotnet-Webapp -Dsonar.projectKey=Dotnet-Webapp"
                }
            }
        }
        stage("quality gate") {
            steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
                }
            }
        }
        stage("TRIVY File scan") {
            steps {
                sh "trivy fs . > trivy-fs_report.txt"
            }
        }
        stage("Docker Build & tag") {
            steps {
                script {
                    withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                        sh "make image"
                    }
                }
            }
        }
        stage("TRIVY") {
            steps {
                sh "trivy image zeeshan321/dotnet-monitoring:latest > trivy.txt"
            }
        }
        stage("Docker Push") {
            steps {
                script {
                    withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                        sh "make push"
                    }
                }
            }
        }
    }
}

When all stages in docker are successfully created then you will see the result You log in to Dockerhub, and you will see a new image is created

Step 9 — Deploy the image using Docker Add this stage to your above pipeline syntax

stage("Deploy to container") {
    steps {
        script {
            sh "docker run -itd --name dotnet -p 5000:5000 zeeshan321/dotnet-monitoring:latest"
        }
    }
}

(Add port 5000 to Security Group)

You will see the Stage View like this,

And you can access your application on Port 5000. This is a Real World Application that has all Functional Tabs.

Step 10 — Access the Real World Application

Step 11 — Kubernetes Set Up

Install Kops and Kubectl on Jenkins machine as well.

curl -Lo kops https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-linux-amd64
chmod +x kops
sudo mv kops /usr/local/bin/kops
curl -LO https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl 
chmod +x kubectl
sudo mv kops /usr/local/bin/kubectl

Create one IAM user and give full admin access
Login with AWS CLI and install AWS CLI using

apt install awscli -y

Give aws configure and give details (AWS Access Key ID,Secret Access Key,region and output format)

Now we need to export our access key and secret key

export AWS_ACCESS_KEY_ID=$(aws configure getaws_access_key_id)
export AWS_SECRET_ACCESS_KEY=$(aws configure get aws_secret_access_key)

Next create S3 bucket and enable versionong

aws s3 mb s3://apr07bucket - region ap-south-1
#mb=make bucket
aws s3api put-bucket-versioning — bucket sep16bucket — versioning-configuration Status=Enabled

Next create sshkey for machines ssh-keygen
id_rsa=private key , id_rsa.pub=public Key

We are now going to give name for cluster by using DNS or gossip- based cluster

DNS:

export NAME=aws.zeeshan.com
export KOPS_STATE_STORE=s3://apr07bucket

Or

Gossip-based: free domain shared by K8’S

For a gossip-based cluster, make sure the name ends with k8s.local. For example:

export NAME=zeeshan.k8s.local
export KOPS_STATE_STORE=s3://apr07bucket Creating Cluster now

Control Plane-master

Node-worker To edit node:

kops edit ig - name=zeeshan.k8s.local nodes-ap-south-1b
ig=instance group

To edit control plane:

kops edit ig - name=zeeshan.k8s.local control-plane-ap-south-1b

Update cluster:

kops update cluster - name zeeshan.k8s.local - yes –admin

List clusters: kops get cluster

Validate Cluster:kops validate cluster

List Nodes: kubectl get nodes — show-labels

Now Give this command in CLI

cat /root/.kube/config

Copy the config file to Jenkins master or the local file manager and save it as secret-file.txt

Now, goto the Master Node

Install Kubernetes Plugin, Once it’s installed successfully

goto manage Jenkins → manage credentials → Click on Jenkins global → add credentials

Install Kubernetes Plugin, Once it’s installed successfully

The final step to deploy on the Kubernetes cluster, add this stage to the pipeline.

stage('Deploy to k8s') {
    steps {
        dir('K8S') {
            withKubeConfig(
                credentialsId: 'k8s',
                serverUrl: '',
                caCertificate: '',
                clusterName: '',
                contextName: '',
                namespace: '',
                restrictKubeConfigAccess: false
            ) {
                sh 'kubectl apply -f deployment.yaml'
            }
        }
    }
}

Before starting a new build remove Old containers. kubectl get svc

#copy service port <worker-ip:svc port>

< http://13.235.69.18:31283/>

Step 12 — Terminate the AWS EC2 Instance

The complete pipeline script

pipeline {
    agent any
    
    tools {
        jdk 'jdk17'
    }
    
    environment {
        SCANNER_HOME = tool 'sonar-scanner'
    }
    
    stages {
        stage('Clean Workspace') {
            steps {
                cleanWs()
            }
        }
        
        stage('Checkout From Git') {
            steps {
                git branch: 'main', url: 'https://github.com/zeeshancli/Dotnet'
            }
        }
        
        stage('Sonarqube Analysis') {
            steps {
                withSonarQubeEnv('sonar-server') {
                    sh "$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Dotnet-Webapp -Dsonar.projectKey=Dotnet-Webapp"
                }
            }
        }
        
        stage('Quality Gate') {
            steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
                }
            }
        }
        
        stage('TRIVY File Scan') {
            steps {
                sh "trivy fs . > trivy-fs_report.txt"
            }
        }
        
        stage('Docker Build & Tag') {
            steps {
                script {
                    withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                        sh "make image"
                    }
                }
            }
        }
        
        stage('TRIVY Image Scan') {
            steps {
                sh "trivy image zeeshan321/dotnet-monitoring:latest > trivy.txt"
            }
        }
        
        stage('Docker Push') {
            steps {
                script {
                    withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                        sh "make push"
                    }
                }
            }
        }
        
        stage('Deploy to Container') {
            steps {
                script {
                    sh "docker run -itd --name dotnet -p 5000:5000 zeeshan321/dotnet-monitoring:latest"
                }
            }
        }
        
        stage('Deploy to Kubernetes') {
            steps {
                dir('K8S') {
                    withKubeConfig(
                        credentialsId: 'k8s',
                        serverUrl: '',
                        caCertificate: '',
                        clusterName: '',
                        contextName: '',
                        namespace: '',
                        restrictKubeConfigAccess: false
                    ) {
                        sh 'kubectl apply -f deployment.yaml'
                    }
                }
            }
        }
    }
}

Here is the GitHub for this project

https://github.com/zeeshancli/Dotnet