Continuous integration with Jenkins and Github Release
Introduction
Github Release is a great way to publish your software. It lets you create releases with notes and assets such as binary files, and works flawlessly with git tags to fit perfectly into your typical git workflow. However, this process is not automated, even less when your project needs to be compiled. Working with a continuous integration tool is therefore a necessity. I personally really like Jenkins, and felt the need to automate my release process for my new projects on Github, but I faced difficulties interfacing it with Github Release as, at the time of writing, no easy option exists to manage releases from Jenkins, and I couldn’t find much information about such automation when searching on Google. So I thought it would be a good idea to share a simple guide to get started with Github Release and Jenkins.
The process is the following:
- Automatically trigger a build on Jenkins when a new tag is created
- Build the project at the given tag
- Create a new release and upload the artifacts using the Github API
Prerequisites
In this guide, I assume that you already have a Github account and a Jenkins install ready. Your build server will need one more tool: curl, which we’ll use to create a release and upload the artifacts to using the Github API.
Setting up Github
To interact with the Github API, Jenkins will need to authenticate using an access token which we will create. From Github, go to Settings -> Developer settings ->Personal access tokens. Generate a new token. We will only check the permissions Jenkins needs for this project.
Copy and paste the generated token somewhere. You will need it later, and you won’t be able to retrieve it again once the popup is closed.
The next part is to create a webhook on your repository. The idea is that Github will send an HTTP request to Jenkins every time something is pushed. Go to repository on Github -> Settings -> Webhooks and create a new webhook.
Leave the “secret” field empty, and check “Just the push event”. Replace “<JENKINS_HOST>” with the ip or domain name of your Jenkins host.
As we only want to build when a tag is created, we could be tempted to use the “create” event that you can see when selecting “Let me select individual events”, but it’s the wrong way because Jenkins and its Git plugin only trigger builds on the “push” event.
We can use the “push” event because, as stated in the documentation:
Branch pushes and repository tag pushes also trigger webhook push events.
This is an important detail that I missed. Even if the hook is sent for every push, Jenkins won’t always build because we will configure it to only build on new tags, so don’t worry about builds getting triggered when you don’t want to.
Setting up Jenkins
The very first step is to install the Github plugin if it’s not already the case. Go to Manage Jenkins -> Manage Plugins -> Available. Type “Github Plugin” in the search bar, install and restart.
Let’s create a new project. Click “New item” in the left menu, select “Freestyle project” and give a name to your project, ideally the same name as the git repository.
Configuring the project
First, in “General”, check “Github project” and type your repository’s URL.
In “Source code Management”, select “Git”. Create new credentials if needed. These are the ones Jenkins will use to pull.
Click “Advanced” and fill “refspec” with the following:
+refs/tags/*:refs/remotes/origin/tags/*A refspec controls the remote refs to be retrieved and how they map to local refs. Here, we only need tags so we set the refspec to tags only.
In “Branch specifier”, type the following:
refs/tags/*This specifier tells Jenkins to only build on new tags, no matter how the tag is named.
In “Build triggers” , check “GitHub hook trigger for GITScm polling”.
This step is optional.
We canto tell Jenkins to archive the artifacts generated by the build. Scroll down to the bottom of the page and add the Post-build action “Archive the artifacts”. Type the name of the files you want to archive. These artifacts will be available to download from the build info on Jenkins.
Create the build script
We’re almost there ! The last step is to create the build script. The process is simple:
- Build the artifacts
- Create a release on Github
- Upload the artifacts to Github
Go to the “Build” section and add a build step “Execute shell”. In the text area, type the following, making the changes needed so it suits your project:
# Build
# "make all" for example# Publish on github
echo "Publishing on Github..."
token="<YOUR GITHUB ACCESS TOKEN>"# Get the last tag name
tag=$(git describe --tags)# Get the full message associated with this tag
message="$(git for-each-ref refs/tags/$tag --format='%(contents)')"# Get the title and the description as separated variables
name=$(echo "$message" | head -n1)
description=$(echo "$message" | tail -n +3)
description=$(echo "$description" | sed -z 's/\n/\\n/g') # Escape line breaks to prevent json parsing problems# Create a release
release=$(curl -XPOST -H "Authorization:token $token" --data "{\"tag_name\": \"$tag\", \"target_commitish\": \"master\", \"name\": \"$name\", \"body\": \"$description\", \"draft\": false, \"prerelease\": true}" https://api.github.com/repos/<USERNAME>/<REPOSITORY>/releases)# Extract the id of the release from the creation response
id=$(echo "$release" | sed -n -e 's/"id":\ \([0-9]\+\),/\1/p' | head -n 1 | sed 's/[[:blank:]]//g')# Upload the artifact
curl -XPOST -H "Authorization:token $token" -H "Content-Type:application/octet-stream" --data-binary @artifact.zip https://uploads.github.com/repos/<USERNAME>/<REPOSITORY>/releases/$id/assets?name=artifact.zip
You can find the Github API documentation here.
Conclusion
Publishing releases on Github when a tag is pushed and using Jenkins is not that hard but not possible natively in the Jenkins Github plugin. I hope this guide helped you! Enjoy your automated releases !
