Continuous Deployment via GitLab, Jenkins, Docker and Slack
In this blog, we will do whole Continuous Deployment process step by step with following technologies: Spring Boot, GitLab, Jenkins,Docker and Slack. We will first create our sample spring boot app with its Unit & Integration tests, then will push it to the GitLab. After pushing the code, Jenkins pipeline will automatically receive code by web hook, and run the tests. If tests pass without any error, Jenkins will build the code and will deploy code to server via docker. Then finally it will send the docker image with its own snapshot to the docker hub registry. If any of the steps in pipeline fails, it will notify the user via Slack.
Before starting to continuous deployment process, let me show you to the below Continuous Deployment diagram that we will do in this blog.
In our scenario, we need Spring Boot app. You can clone sample spring boot app from my github repository: https://github.com/onedaywillcome1/ContinuousIntegrationAndContinuousDeliveryApp.git
It is Hello World Spring Boot-Maven app which owns 1 unit test & 1 integration test.
First, Make a setup in GitLab integrations menu to trigger Jenkins.
GitLab is now ready to trigger Jenkins when all setup is made in Jenkins.
Next, Let’s install Jenkins, Docker and some other requirements. Here is the final step of launched AWS EC2 instance. Instance public ip is: 52.11.94.229
Now let’s ssh to instance and install following requirements:
# ssh to instance
ssh -i ~/.ssh/jenkinskeypair.pem ec2-user@52.11.194.229# Update packages
sudo yum update -y# Install Git
sudo yum install git -y# Download Jekins repo
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo# Import jenkins key
sudo rpm — import https://jenkins-ci.org/redhat/jenkins-ci.org.key# Install Jenkins
sudo yum install jenkins -y# Start Jenkins
sudo service jenkins start# Enter to home directory
cd ~# Download Java 8
wget — no-cookies — no-check-certificate — header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept securebackup-cookie" "http://download.oracle.com/otn pub/java/jdk/8u60-b27/jdk-8u60-linux-x64.rpm"# Install Java 8
sudo yum localinstall jdk-8u60-linux-x64.rpm# Check the java version
java -version# Copy JAVA_HOME to bash_profile
echo "export JAVA_HOME=/usr/java/jdk1.8.0_60/jre" >> ~/.bash_profile# Run the bash_profile
source ~/.bash_profile# Check JAVA_HOME is successfully installed as environment variable
env# Download Maven
wget http://mirror.olnevhost.net/pub/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz# Untar maven gz file
tar xvf apache-maven-3.0.5-bin.tar.gz#Move apache file to /usr/local dir
sudo mv apache-maven-3.0.5 /usr/local/apache-maven# Copy below Maven commands to bash_profile
echo "export M2_HOME=/usr/local/apache-maven" >> ~/.bash_profile
echo "export M2=$M2_HOME/bin" >> ~/.bash_profile
echo "export PATH=$M2:$PATH" >> ~/.bash_profile# Run the bash_profile
source ~/.bash_profile# Check Maven commands successfully installed as env. variable
env# Install Docker
sudo yum install -y docker# Start docker
sudo service docker start# Add docker user to ec2-user and jenkins
sudo usermod -a -G docker ec2-user
sudo usermod -a -G docker jenkins# Check docker info
sudo docker info# Exit from instance
exit# Ssh again
ssh -i ~/.ssh/jenkinskeypair.pem ec2-user@52.11.194.229# Now we are able to check info without sudo
docker info# Finally restart jenkins & docker
sudo service docker restart
sudo service jenkins restart
Jenkins is now up and running. You can access via http://52.11.194.229:8080 copy the Jenkins admin password as shown below and unlock jenkins and install suggessted plugins
Install GitLab & Slack plugin in Jenkins
Go to http://52.11.194.229:8080/pluginManager/available and select Gitlab plugin and install without restart.(In your case, public ip will change, don’t forget it)
After installing slack & gitlab plugin, we can do following step ( optional). Slack plugin needs outgoing webhook config. If we leave it blank, jenkins will give an exception. So, to fix this problem, we should do below steps:
sudo mkdir /var/lib/jenkins/init.groovy.d
sudo vi /var/lib/jenkins/init.groovy.d/disable-slack-webhooks.groovyThe copy below groovy script in disable-slack-webhooks.groovy file
import jenkins.model.Jenkins
import hudson.model.RootActiondef j = Jenkins.instance;
def removal = { lst ->
lst.each { x ->
if(x.getClass().name.contains("slack.webhook")) {
lst.remove(x)
}
}
}removal(j.getExtensionList(RootAction.class))
removal(j.actions)
Give Jenkins ownership to init.groovy.d directory and restart jenkins again:
sudo chown jenkins:jenkins -R /var/lib/jenkins/init.groovy.d
sudo service jenkins restartLet’s configure jenkins now:
Go to http://52.11.194.229:8080/configure and Configure Java_home and M2_HOME as environment variables and fill the Slack notification settings.
Go to http://52.11.194.229:8080/configureTools and configure the M2 path
Now we have configured fully up and running Jenkins machine. Now, we can create our Pipeline job and create Pipeline script. Select “Build when a change is pushed to GitLab” button because we will push our code to Gitlab, and Gitlab will trigger
Copy below pipeline script to Pipeline job. In preparation stage, Jenkins will clone sample spring boot app from GitLab repository and will run tests in “Test” stage. If tests pass, it will switch to “Build” stage, then results and finally deployment stages.
node {
def mvnHome
stage(‘Preparation’) { // for display purposes
git ‘git@gitlab.com:<myRepo>/ContinuousIntegrationAndContinuousDeliveryApp.git'
mvnHome = tool 'M2'
}
stage('Test') {
try {
sh "'${mvnHome}/bin/mvn' test"
} catch (e) {
notifyStarted("Tests Failed in Jenkins!")
throw e
}
}
stage('Build') {
try {
sh "'${mvnHome}/bin/mvn' clean package -DskipTests"
}catch (e) {
notifyStarted("Build Failed in Jenkins!")
throw e
}
}
stage('Results') {
try{
archive 'target/*.jar'
}catch (e) {
notifyStarted("Packaging Failed in Jenkins!")
throw e
}
}
}
stage('Deployment') {
try{
sh '/var/lib/jenkins/workspace/Pipeline/runDeployment.sh'
}catch (e) {
notifyStarted("Deployment Failed in Jenkins!")
throw e
}
}
notifyStarted("All is well! Your code is tested,built,and deployed.")
}def notifyStarted(String message) {
slackSend (color: '#FFFF00', message: "${message}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")
}
Now go to /var/lib/jenkins/workspace/Pipeline directory. Create runDeployment.sh and deployment directories and put Dockerfile in deployment directory.
cd /var/lib/jenkins/workspace/Pipeline
mkdir deployment#Copy below Dockerfile
sudo vi deployment/Dockerfile
FROM java:latestEXPOSE 8090
ADD app.jar myApp.jar
ENTRYPOINT ["java","-jar","myApp.jar"]# Create runDeployment.sh and copy below script with your docker username and password.
sudo vi runDeployment.sh
#!/bin/bash -execho "Deploying app.jar to docker folder"
packageName=`ls target/continuousintegrationandcontinuousdeliveryapp*.jar`
versionid=`echo $packageName | awk -F "-" '{ print $2}'`
versionname=`echo $packageName | awk -F "-" '{ print $3}' | awk -F "." '{ print $1}'`
version=`echo $versionid-$versionname`
echo "version: $version"
cp -r $packageName deployment/app.jardockerImageName=onedaywillcome/myapp
dockerpid=`docker ps -a | grep $dockerImageName | grep "Up" | awk -F " " '{ print $1 }'`
if [[ $dockerpid != "" ]];then
docker kill $dockerpid
docker rm $dockerpid
fi
docker build -t $dockerImageName deployment/.
docker run -d -p 8090:8090 $dockerImageNamedockerImageId=`docker images | grep $dockerImageName | grep latest | awk -F " " '{print $3}'`
docker tag $dockerImageId $dockerImageName:$version
docker login -u <DOCKER_USERNAME> -p <DOCKER_PASSWORD>
docker push $dockerImageName:$version#Give jenkins ownershipt to deployment directory and runDeployment.sh
sudo chown jenkins:jenkins runDeployment.sh
sudo chmod 775 runDeployment.shsudo chown jenkins:jenkins -R deployment
Finally, all is done! When you now push code to GitLab. GitLab will trigger Jenkins, then Jenkins will first run the tests, build, deploy in server and will send snapshot to Docker Registry, besides that it will notifies you when anything happens in pipeline via Slack
Our Spring app is now up and running:
And Docker images with its own snapshot sent to Docker hub registry:
Of course, my post which explains CD is not real-world example. In real-world example, there will be more features and security. But, it can be beneficial for developers who is new and try to understand Continuous Deployment flow. Please like the post, if you find it useful. ;) See you in next post!