Automated Daily Backups of Jenkins Master to Amazon S3 Bucket and Restore into another new Jenkins server.

Contents:

• Introduction
• Automated Daily Backups of Jenkins Master to Amazon S3 Bucket
• Restore into another New Jenkins Server
• Conclusion

Introduction

In modern software development, Jenkins plays a crucial role as an automation server for continuous integration and delivery. Ensuring the security and availability of your Jenkins master configuration is vital. This guide provides a concise, step-by-step approach to automate daily backups of your Jenkins master to an Amazon S3 bucket, enhancing reliability.

By installing the AWS CLI, creating an S3 bucket, configuring a Jenkins job, and performing regular backups, you’ll establish a robust mechanism for the continuity and recoverability of your Jenkins environment. This setup ensures the security and availability of essential data, including configuration and job settings. Follow these steps to achieve a fully operational backup solution.

Automated Daily Backups of Jenkins Master to Amazon S3 Bucket

Step 1: Install AWS CLI on your EC2 Instance

1. Log in to your AWS account and navigate to the EC2 dashboard.
2. Connect to the instance where Jenkins is installed using SSH.
3. Install AWS CLI using the following command:

sudo apt install awscli

4. Verify the installation:

aws --version

Step 2: Create an S3 Bucket

1. In your AWS account, navigate to the S3 dashboard.
2. Create a new bucket with a unique name.
3. Go with the default settings for now.
4. Review and create the bucket.

Step 3: Attach IAM Role to Instance

Ensure that your IAM role is attached to the instance to allow successful uploads to S3.

Step 4: Create a Jenkins Job

1. In Jenkins, click on ‘New item’ in the left navigation panel.
2. Name the job as ‘jenkins-backup’ and choose ‘Freestyle project’ for the project type.

Step 5: Configure Jenkins Job

1. Build Triggers > Build Periodically:

• Set up a CRON job to run the backup every day at midnight: 0 0 * * *.

2. Build > Execute Shell:

• Add the following script to perform the backup:

echo 'tar $JENKINS_HOME directory'
set +e
tar -cvf jenkins_backup.tar -C "$JENKINS_HOME" .
exitcode=$?
if [ "$exitcode" != "1" ] && [ "$exitcode" != "0" ]; then
  exit $exitcode
fi
set -e
echo 'Upload jenkins_backup.tar to S3 bucket'
aws s3 cp jenkins_backup.tar s3://your bucket name/
echo 'Remove files after successful upload to S3'
rm -rf *

3. Save the job configuration.

Step 6: Test the Jenkins Job

1. Ensure the job is set to run at midnight every night.
2. Manually build the job to verify its functionality.

Step 7: Verify Backup in S3 Bucket

1. Check the ‘Console Output’ to understand how the script ran.
2. Visit your S3 bucket and confirm the presence of the ‘jenkins_backup.tar’ file.

Step 1: Create a New EC2 Instance

Create a new EC2 instance with the desired specifications. Make sure to associate the instance with the appropriate IAM role and security groups.

Step 2: Install Jenkins

Install Jenkins on the new EC2 instance. Skip the steps for installing plugins and logging in since the intention is to use the previous Jenkins server login details.

Step 3: Install AWS CLI

Install AWS CLI on the Jenkins server using the following command:

sudo apt-get install awscli # For Ubuntu/Debian

Step 4: Modify IAM Role

Modify the IAM role associated with the new EC2 instance to ensure it has the necessary permissions to access the S3 bucket containing the Jenkins backup.

Step 5: Stop Jenkins Server

Stop the Jenkins server using the following command:

sudo systemctl stop jenkins

Step 6: Download Backup from S3 Bucket

Download the backup file from the Amazon S3 bucket using the AWS CLI:

aws s3 cp s3://s3bucketname/backupname jenkins_backup.tar

Step 7: Remove Jenkins Folder

Remove the existing Jenkins folder to prepare for the restoration:

sudo rm -rf /var/lib/jenkins/

Step 8: Restore Jenkins Home Directory

Restore the Jenkins home directory from the backup file:

tar -xvf Jenkins_backup.tar -C / (put path /var/lib/jenkins)

Step 9: Start Jenkins Service

Start the Jenkins service using the following command:

sudo systemctl start jenkins

Conclusion

You have now successfully set up automated daily backups of your Jenkins instance to an Amazon S3 bucket. The Jenkins job will run every day at midnight, creating a tar.gz file of the ‘jenkins_home’ directory, uploading it to the S3 bucket, and removing local files after a successful upload.

Following these steps should successfully restore Jenkins to a new server using a backup from an S3 bucket. Ensure that all configurations are correct and that the IAM role has the necessary permissions for a smooth restoration process.