The reusable microservice pipeline pattern

Create a repo called common-pipelines, that contains a collection of reusable Jenkinsfiles.

Perhaps you are using more than one tech stack, so create one for each microservice tech stack e.g. spring-boot-microservice.groovy, node-js-microservice.groovy, etc.

Your spring-boot-microservice.groovy common Jenkinsfiles may have this sort of shape:

#!groovy
def runJenkinsfile(){
    node{
        stage('checkout'){ ... }
        stage('build'){ ... }
        ...
        stage('check code coverage'){ ... }
        stage('early functional test'){ ... }
        ...
        stage('cut release'){ ... }
        stage('publish artifacts'){ ... }
        ...
        stage('deploy to CI'){ ... }
        stage('integration test'){ ... }
        ....
        stage('deploy to production'){ ... }
        stage('smoke test'){ ... }
    }
}
return this;

Now that you have a reusable common Jenkinsfiles for spring boot microservices, you can point to it from each of your Jenkinsfiles.

Here’s an example of what the Jenkinsfile in each spring boot microservice repo could look like to achieve this:

#!groovy
def gitProject = 'your-git-project'
def gitRepo = 'your-git-repo'
def gitPath = 'ssh://git@your.git.server.com:22/' + gitProject + '/' + gitRepo
def gitBranch = 'master'
def jenkinsPipeline = 'spring-boot-microservice' // ".groovy" extension will be added automatically
def jenkinsCredentialsId = 'your-jenkins-credentials-id'
def jenkinsNode = '' // Specifies a node to be used for checkout, default value: empty string (runs on any node)
def pipeline

stage('load pipeline'){ 
    pipeline = fileLoader.fromGit(
        jenkinsPipeline,
        gitPath,
        gitBranch,
        jenkinsCredentialsId,
        jenkinsNode
    )
}

pipeline.runJenkinsfile()

That’s it. You’ll now be able to control the pipeline from a single place.

Remember that you can get hold of the GitHub Organisation Folder Plugin, or the equivalent for Bitbucket. Both will scan specified git projects for repos containing Jenkinsfiles, and build your pipelines from code.

I recommend that you consider some of these extension points beyond the base pattern:

1. Consider creating a common-pipelines repo for each of your scrum teams to help keep your teams loosely coupled, yet highly aligned — balancing flexibility (implementation) with consistency (pattern).
2. Consider versioning your common pipelines and release them as immutable artifacts to give your scrum teams the option to lock to specific pipeline versions as things evolve (e.g. spring-boot-microservice-2.0.0.groovy). This will give your scrum teams independence, such as being able to migrate to newer versions when they decide.
3. Consider decomposing down even further, creating common pipeline stages that can be optionally pulled in by your scrum teams (mixins).