Game Tech Tutorial
Published in

Game Tech Tutorial

How to Scale Master and Slave Architecture using Jenkins? (CI/CD: Part 2)

This article will detail how to set up Master-Slave Architecture on Jenkins for high availability and scalability.

In the Master-Slave architecture of Jenkins, it is very common to have one master connected to several slaves.

Suppose a master node crashes due to an unexpected hardware failure. Your CI/CD service may be interrupted at some time.

In this post, you will learn

  • How to make the master node highly available for Master-Slave architecture?
  • How to scale the capacity of slave nodes?

How to make the master node highly available for Master-Slave architecture?

You can set up DNS services like Amazon Route 53 which can easily achieve failover architecture for Jenkins and ensure high availability.

It’s recommended to design the following Master-
Slave architect to integrate Github and Jenkins with Amazon Route 53.

You can refer below to understand Master-Slave architecture for details.

How to Setup Master and Slave Architecture using Jenkins? (CI/CD)

Flow:

Developer → GitHub → Amazon Route 53 → Master-Active and Master- Standby node → Multiple Slave nodes

  • When a developer pushes to Github, Github will send Push events to Master-Active or Master-Standby machine via Amazon Route 53.
  • Amazon Route 53 can determine available nodes to direct traffic to either master-active or master-standby.
  • If Master-Activeve dies, Route 53 can automatically switch traffic to the available node for Master-Standby.
  • To scale the capacity, you can add slave nodes behind the master node when needed via SSH connection. ( Refer to How to Setup Master and Slave Architecture using Jenkins? for details )
  • You need to set up the master-active to connect master-standby so that master nodes can failover each other to ensure high availability by the following settings in Pipeline.

With this setting, Jenkins’s Pipeline can select available nodes on Master-Active or Master-Standby to execute jobs.

How to Scale Capacity on Slave Node?

This is tricky I would like to share some tips with you because this is not easy to find.

You may know master node in the Master-Slave architecture of Jenkins can assign a label to determine which slave node can run specific jobs.

However, how to smoothly dispatch multiple build evens to slave nodes?

The trick is you can assign multiple slave nodes with the same label in the pipeline.

Or you can assign different labels for different slave nodes.

Is it possible to make the slave node run multiple jobs at the same time?

Yes, remember you need to uncheck below to allow your pipeline to run parallel jobs.

Summary

In this post, you have learned

  • how to make master nodes of Jenkins in high availability via Amazon Route 53.
  • how to scale capability by running current jobs on a single node or dispatch jobs to slave nodes by assigning the same label.

This can easily achieve horizontal scaling by adding new slave nodes with the same label in Jenkins’s pipeline for your CI/CD backend.

Hope this tutorial will be helpful for you.

Thank you for reading!

If you liked this story, please show your support by 👏 this story or subscribe to our email to get the latest tutorials in the future.

You might be interested in

--

--

High-quality technical tutorials: AWS, GCP, Azure, Unity, SDK, Security, DevOps, and more.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Eric Wei

Senior Full Stack Engineer & Solution architecture | AWS, GCP, Azure | Cloud, Unity Game Development, SDK, DevOps, and more.