Simple, Repeatable & Free: An Open Source Software Delivery Pipeline (Part II)

In the last installment of this series (https://medium.com/stainless-ai/simple-repeatable-free-an-open-source-software-delivery-pipeline-part-i-4ae89e7a6478), we

• Reviewed some DevOps concepts;
• Discussed semantic versioning for our software packages;
• Introduced a custom Gitflow and;
• Introduced a Jenkins shared library for automating semantic versioning of our builds

In this installment we’re going to take a closer look at the jenkins-semci library and walk through using it in a simple project.

Photo by Brooke Cagle on Unsplash

Components

The pipeline we’ll describe here is designed to be as simple as possible while providing a high degree of automation and policy enforcement. It’s composed of several components, most of which are probably already in use in your organization in some form or another.

1. Source Repository: We’ll use GitHub for managing our source code. Our solution relies on repository tagging to automate versioning, so if you have a different repository, it must allow tagging. To use our OSS plugins, it must also implement the Git protocol.
2. Build Server: We’ll use Jenkins (https://www.jenkins.io) for automating our builds. Jenkins is a free, mature, open source build server written in Java. It provides a feature-rich plugin ecosystem and hooks for writing build scripts.
3. The jenkins-semci plugin (https://github.com/stainlessai/jenkins-semci). This plugin provides a shared library for Jenkins to automate artifact versioning using Semantic Versioning, Git tags and branches.

Jenkins supplies a tutorial for creating multibranch pipelines, and we’re going to piggyback on that tutorial, adding automated versioning capability to the sample project provided there. The tutorial is here:

End-to-End Multibranch Pipeline Project Creation

NOTE: There are some changes we need to make to the stock tutorial to demo jenkins-semci. The docker agent we use to build must have a git client installed. The agent described in the tutorial doesn’t by default. To do this, we’ll add the following Dockerfile to the project, called “Dockerfile.build”:

FROM node:6-alpine
RUN apk add -U git

Next, we’ll change the agent definition in our build file to use this Dockerfile to build our agent. We’ll modify our Jenkinsfile as follows:

pipeline {
    agent {
        dockerfile {
            filename 'Dockerfile.build'
            args '-p 3000:3000 -p 5000:5000'
        }
    }
... FILE CONTINUES ...

If you already have a Jenkins project you want to enhance with the jenkins-semci plugin, you can skip the tutorial. Otherwise, this article assumes you’ve gone through the tutorial with the above modifications, and have Jenkins set up with the multibranch pipeline created. After running the tutorial, you have 3 branches: development, master, and production, each with a Jenkinsfile that has the stages “build,” “test,” “Deliver for development,” and “Deploy to production.”

First we have to configure the shared library in Jenkins. This will instruct Jenkins to pull the library code directly from Github when the jobs run. This is achieved on the Jenkins job configuration page, which is accessible from the “Configure” menu item of the job page in Jenkins classic, and by clicking the cog icon on the job page in Blue Ocean.

Configure job icon on the Job page

Configure job on the Blue Ocean page

Once on the configure page, scroll down to the “Pipeline Libraries” section, and click “Add”

Setting up the jenkins-semci shared library

We’ll name the library “jenkins-semci,” although you can name it whatever you want as long as you use the same name in the corresponding .@Library tag in the pipeline file (see below).

We’ll use the “GitHub” option under “Source Code Management,” and fill in the “Repository HTTPS URL” value. It’s a public repository, so no credentials are needed.

Setting up the repository URL for the shared library

We’re now going to open that pipeline file and install the jenkins-semci plugin by adding the lines below:

@Library('jenkins-semci')
import ai.stainless.jenkins.ReleaseManager
def releaseManager = new ReleaseManager(this)
pipeline {

... FILE CONTINUES ...

The last thing we need to do is to ensure that our project pulls down tags when it checks out revisions to build. This is accomplished by adding an “Advanced Clone Behavior” in the project configuration under the Branch Sources panel of the configuration:

Configuring our job to fetch tags on checkout

The ReleaseManager class assumes the branch that produces release is named “master.” To change this to “main,” or in the case of the tutorial, “production,” we set the masterBranch property of the ReleaseManager to the value we want:

releaseManager.masterBranch = 'production'

Now, if we run a build on the production branch, two things will happen:

1. The ReleaseManager will check to see if the commit hash of the build matches the commit hash of a release tag. If no match is found, an error is thrown. To resolve, tag the commit you want to release with the desired version and ensure that commit is on the master branch.
2. The computed version will reflect the tag of the commit.

If we run a build on any other branch, the commit check is skipped and the build is named with the branch name and -SNAPSHOT in the pre-release.

To try it out in our tutorial build, we can add a “Version” stage to the build and print out the computed version. Note that we can all the releaseManager.artifactVersion() method from any stage, and even pass the string to other calls, like Docker tags, or build scripts. There are other methods as well that just return the project name appended to the version if we wanted to write a complete filename.

stages {
        stage('Version') {
            steps {
                print releaseManager.artifactVersion()
            }
        }
... FILE CONTINUES ...

After running the ‘development’ branch build, we’ll see

Output from successful call to jenkins-semci library

The version 0.0.1 indicates we haven’t added any tags, and this is build #15. It’s not on the ‘production’ branch, so it’s labeled with prerelease development-SNAPSHOT. If we ran this on the master branch, we’d see a prerelease of master-SNAPSHOT.

Let’s build a release! First, add a valid semantic tag to the repository prefaced with v, e.g.

git checkout production
git merge development // merge in our recent changes
git tag -a v1.2.3 -m "Testing tags"

Now run the production branch build in Jenkins, and releaseManager.artifactVersion() returns a version corresponding to your new tag:

Newly-minted version 1.2.3

Finally, run the development branch build again. You’ll see the semantic version has changed to reflect the fact that version 1.2.3 was released while you were working on your branch, so your SNAPSHOT versions now start at version 1.3.0

Dynamic version updating!

The library also supports multiple projects in a single repo using an at ‘@’ prefix for discriminating between projects. More details and documentation can be found at the shared library README. Let us know what you think, or if you’re successfully using the library! PRs open for updates and changes!

stainlessai/jenkins-semci