Blue - Green Deployment with Jenkins
Recently, we have started to use a continuos delivery pipeline and I want to share my experience about our transition from manual deployment ( aka. copy-paste) to continuos delivery.
How Was It Before?
We have 6 different applications for now. And 2 of them have multiple nodes. So, if someone needs to deploy new features to live, he or she needs to copy files to every application. It’s really really hard to do.
Eventually, we’ve decided to setup a continuous delivery pipeline though it’s too late.
I will try to explain only web and api deployment in this post.
Blue — Green Deployment
A blue/green deployment is a change management strategy for releasing software code. Blue/green deployments, which may also be referred to as A/B deployments require two identical hardware environments that are configured exactly the same way. While one environment is active and serving end users, the other environment remains idle.
If problems occur after the switch, traffic can be directed back to the idle configuration that still runs the original version.
This is the most important point; If you setup a blue-green deployment pipeline, you will be able to test in live!
Continuous Delivery
Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time.
Continuous Delivery doesn’t mean every change is deployed to production ASAP. It means every change is proven to be deployable at any time.
Continuous Delivery vs Continuous Deployment
We didn’t want to setup a continuous deployment pipeline. We want to control deployment manually.
Tool Options
After deciding continuos delivery, we had a lot of options and we studied most of them. I will give you some options.
- Web site: https://jenkins.io/
- Open Source — Hudson Company
- Blue Ocean Plugin
- Platform Independent — Java Based
- Good community support
- A lot of plug-ins (1400 for now)
- Web site: https://www.jetbrains.com/teamcity/
- JetBrains Company
- Platform Independent -Java Based
- Free Usage has limitations(3 build agents vs)
- Configuration options
- 200 plugins
- Web site: https://www.atlassian.com/software/bamboo
- Atlassian Company
- Free to try for 30 days — $5 per user per month (according to our configuration)
- Web site: https://www.gocd.org/
- Open source — ThoughtWorks company
- Platform Independent — Java Based
- Server-Agent relation
- 50–100 Plugins
- Web site: https://travis-ci.com/
- $249 Per month (according to our configuration)
- Focus on writing code. Let Travis CI take care of running your tests and deploying your apps.
- No .net Support
After long researches, blue ocean plug-in made it easier for us to choose jenkins.
Jenkins Pipeline and Blue Ocean
Jenkins Pipeline (or simply “Pipeline” with a capital “P”) is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.
Jenkinsfile Example:
pipeline {
agent any stages {
stage('Build') {
steps {
sh 'make'
}
}
stage('Test'){
steps {
sh 'make check'
junit 'reports/**/*.xml'
}
}
stage('Deploy') {
steps {
sh 'make publish'
}
}
}
}
- Formerly known as workflow.
- Distrubutable!
- Visualized!
- Single and simple groovy script and no more a lot of job configurations
Our Deployment Pipelines
After describing the terms, I can show you our setup.
We didn’t use Jenkins pipeline source control feature. We created “New Pipeline Item” and configured pipeline script in that.
- We have 4 pipelines for now.
- We have email notification at the end of every pipeline.
- I will give you Blue and Green pipelines. Others are almost same.
Blue Pipeline
This is the starting point.
- Deploys api and web site.
- Pulls branch last revision.
- Builds and prepares deployment packages
- Stop Sites(Stop load balancer routing, stop IIS)
- Copy new files
- Start Sites(Start only IIS, not routing)
Green Pipeline
If everything is fine in blue servers, we can now go to green pipeline and live.
- Takes parameter to deploy which version (We prepared packages in blue)
- Activate Blue Sites Routings(Load balancer)
- Stop Sites(Stop load balancer routing, stop IIS)
- Copy new files
- Start Sites(Start only IIS, not routing)
- Copy new files
- Activate Blue Sites Routings(Load balancer)
And we are LIVE! Load balancer now sends requests to all servers (blue and green)
Extra Informations
Deployment Server
We have one dedicated server. It has 4gb ram, Intel Xenon 2 processor CPU and 100gb hdd.
We created a folder and copied and created all necessary things in that.(Batch scripts, status files, source control, PPsExec, Nuget, MsBuild, Created Build Packages in Pipelines)
We have all source files in deployment server. It connects scm with access key. We use bitbucket and mercurial.
Routing
We use F5 for load balancing.
We have abc.txt files in every server. Load balancer reads files at certain intervals. If it sees “OK” in file, sends requests to that server.
We have active and passive files in deployment server. We copy them when necessary(activate or passivate).
IIS Operations
We used PsExec.exe to start-stop-recycle site and application pool.
Batch Scripts
I created a repo in github. You can find all scripts there.
Thank your for reading. If you liked post I would be very happy if you’d click the clap button. Please feel free to contact me. You can reach me on Twitter, or you can leave a comment.
