A Hacky Hacker’s Guide To Hacking Together Jenkins Scripted Pipelines and Getting Them To Do Things, Part 1

Whether you’ve outgrown the bounds of Jenkins freestyle jobs, walked into existing Jenkins pipeline infrastructure with no prior experience (high-five me too!), or have decided to implement Jenkins with pipelines from the get-go, something has compelled you to seek out just how to get Jenkins scripted pipelines to do things. Welcome.

I hate to be the bearer of bad news, but the documentation is lacking (and often unspecific as to whether or not its examples are for scripted or declarative pipelines), the book barely grazes the subject (if you’ve found one that has better information please share), and piecing together bits of other peoples’ Github examples and Stack Overflow answers is nothing short of a painful slog. Even then, you’re just as likely (if not more!) to find declarative pipeline examples which, of course, have different syntax and abilities.

What we really need is for an expert to write out, head to tail, all the things one would need to know to build complex and powerful pipelines. The Right Way To Go About Things. But, since we don’t have that, I’m going to do my best to compile what I’ve found and maybe it’ll save you a bit of time in the future. It may not be the best way to get it done, but at least it’s a way to get it done.

The Structure

A Jenkins scripted pipeline is a single piece of Groovy code, in a Jenkinsfile, made up of stages, like so:

#!/usr/bin/env groovy
stage('build') {
node('java8') {
// this is a comment
}
}
stage('test') {
node('testing') {
// this is where testing code goes
}
}
stage('deploy') {
node('deploy') {
// push stuff to aws or something
}
}

A few things: ‘stages’ are just what you call each step of the pipeline, and their names can be whatever you’d like (though preferably something short and descriptive). A ‘node’ is the server where you run your code. If no node is specified, your code will run on the master Jenkins, which people generally don’t want. If your Jenkins setup is already established, it probably has certain nodes set up already (generic Jenkins workers, containers with different operating systems or programming languages installed, etc), otherwise, you’ll want to go ahead and set some up, but that’s way out of our scope here.

You can use the same node over and over, or you can use different nodes for each stage, depending on what they do. Provided you don’t need to switch nodes, you can even have all the stages run on the same node:

node('centos') {
stage('step 1') {
echo 'This is the first stage.'
}
  stage('step 2') {
echo 'This is stage 2, running on the same node as stage 1. By having both stage blocks *inside* the node block, the pipeline runs much quicker.'
}
}

The Guts

So what goes on inside the stages? Well…whatever you’re trying to accomplish. If you’re building code with Apache Maven, it might look sort of like this:

stage('build') {
node('java8') {
sh 'mvn clean deploy -U'
}
}

Maybe you’re trying to run a script from a Github repo:

stage('get code') {
node('jenkins-worker') {
// just by specifying your git credentials and repo, the pipeline knows to check it out and run the next steps from that repo
git credentialsId: '$key', url: 'git@github.com:$usr/$repo.git', branch: "${env.BRANCH_NAME}"
// this is a script in the root of the $repo.git repository
sh './run-me.sh'
}
}

Or interacting with AWS:

stage('deploy') {
node('aws') {
// this is going to be a multi-line 'sh'
sh """
new_stack = "my-aws-cloudformation-stack-name"
aws cloudformation create-stack \
--region us-east-3 \
--capabilities CAPABILITY_IAM \
--stack-name $new_stack
--template-body file://path-to-file-for-stack-template
// the wait is optional, but recommended, so the pipeline doesn't progress until your stack is actually created
aws cloudformation wait stack-create-complete --region us-east-3 --stack-name $new_stack
"""

I’m noticing a theme here…

If you’re wondering why I didn’t say before that everything needs to be run either as sh 'things here' or in an sh block, that’s because there are some things that don’t need to be run that way. It’ s just that a lot of them do.

With that said, that should cover the basics of Jenkins scripted pipeline structure. Next up: input, conditionals and doing multiple things at once.

Did I miss anything? See something incorrect, or that could be done better? Have requests for what else you’d like to see covered? Let me know in the comments!