How to backup Jenkins configuration to Google Cloud Storage

Avi Klaiman
Oct 14 · 5 min read

Recently, I needed to backup a Jenkins server configuration, which is deployed on top of a Kubernetes Cluster on Google Cloud, on a daily basis.

I have searched the web for a solution and found out that the most recommended Jenkins plugin for backing up Jenkins configuration is ThinBackup.

ThinBackup allows you to backup your whole Jenkins configuration and also has a cron like scheduler where you specify when each backup takes place.

The main risk is that these backups are saved locally on the Jenkins server so if something happens to the server — you can say goodbye to your backups. This concern has led me to the decision to backup my Jenkins configuration to Google Cloud Storage.

Following is a step by step guide to create your own automated backup to Google Cloud Storage:

  1. Go to the plugin manager section and install ThinBackup plugin:

After installing the plugin, let’s configure it:

Go to Manage Jenkins and you should see ThinBackup option — click on it:

Choose Settings, put the following configuration and click Save:

You can read more about the plugin options here.

From now on, we will be having a daily backup of our Jenkins server.

Let’s also create our first manual backup by clicking on Backup Now:

2. Install Google Cloud Storage plugin:

We will be using this plugin to connect to Google Cloud Storage.

3. Create a Service Account with Google Cloud Storage write or admin privileges.

4. Create a Google Storage Bucket for your Jenkins backups and make sure the above service account has write or admin permissions for this bucket.

5. Upload the Service Account JSON key you created at step 3 to Jenkins as credentials — you will see a new option called “Google Service Account from private key” in the credential sections in Jenkins:

6. Configure Jenkins master node to run the backup:

Fill the following:

# of executors — 1

Labels — master.

Press Save.

This will enable the Jenkins Server to run the backup pipeline you will create in the next section.

7. Create a new Pipeline:

Let’s configure it to run once a day:

Now let’s write the script which will upload our backups to the remote storage:

Let’s look at the code:

JENKINS_BACKUP_FILE_NAME = sh (
script: '''#!/bin/bash
# find latest backup
thinbackup_path="$JENKINS_HOME/thinBackup"
latest_backup_folder=$(ls -t $thinbackup_path | head -n1)
full_path="$thinbackup_path/$latest_backup_folder"

# pack it up
backup_name="$latest_backup_folder.tar.gz"
tar cvzf $backup_name $full_path
echo ${backup_name}
''',
returnStdout: true
).trim().tokenize().last()

First, we want to find the last backup folder that was created by ThinBackup. After that, we create an archive from that folder and pass the archive name to a Jenkins environment variable (I used the code from the following gist as a starting point: https://gist.github.com/calvinlfer/55db013f17fb8047a753174e73cc5919).

step([$class: 'ClassicUploadStep', credentialsId: 'MyGoogleCloudProjectName',  
bucket: "gs://my-jenkins-backups",
pattern: JENKINS_BACKUP_FILE_NAME])

Next, I used the google cloud storage plugin build step to upload the archive to the bucket. MyGoogleCloudProjectName — stands for the project name on Google Cloud. my-jenkins-backups — stands for the bucket name which will store the backups.

sh '''
rm '''+JENKINS_BACKUP_FILE_NAME+'''
'''

Finally, after uploading the archive, we will delete it.

Here is the full code:

pipeline {
environment {
JENKINS_BACKUP_FILE_NAME = ''
}
agent {
label 'master'
}
stages {
stage('Backing up Jenkins') {
steps {
script {
JENKINS_BACKUP_FILE_NAME = sh (
script: '''#!/bin/bash
# find latest backup
thinbackup_path="$JENKINS_HOME/thinBackup"
latest_backup_folder=$(ls -t $thinbackup_path | head -n1)
full_path="$thinbackup_path/$latest_backup_folder"

# pack it up
backup_name="$latest_backup_folder.tar.gz"
tar cvzf $backup_name $full_path
echo ${backup_name}
''',
returnStdout: true
).trim().tokenize().last()
}
step([$class: 'ClassicUploadStep', credentialsId: 'MyGoogleCloudProjectName',
bucket: "gs://my-jenkins-backups",
pattern: JENKINS_BACKUP_FILE_NAME])
sh '''
rm '''+JENKINS_BACKUP_FILE_NAME+'''
'''
}
}
}
}

Click Save and let’s test our script.

8. Testing our backup script:

Let’s go to our new pipeline and press Build Now:

And that’s it! You should see the following result:

Now you can check the bucket and see your backup there!

Final Notes:

In this article, we learned how to backup our Jenkins configuration on a daily basis and upload the backup to a Google Cloud Storage bucket.

If you want to backup the secrets folder as well, you can check the following link:

https://stackoverflow.com/questions/40999412/jenkins-thinbackup-include-additional-folder

You should consider the security implications though— because if you backup the secrets directory as well, you expose your secrets in case someone is able to access the bucket that stores your backups.

Follow us on Twitter 🐦 and Facebook 👥 and join our Facebook Group 💬.

To join our community Slack 🗣️ and read our weekly Faun topics 🗞️, click here⬇

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇

 by the author.

Avi Klaiman

Written by

Faun

Faun

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade