Jenkins Configuration as Code on AWS
by Harshita Mishra and Prachi Jain
The “as-code” paradigm is the ability to setup and / or restore a complete environment in minutes based on code-driven recipes and automation. Jenkins Configuration as Code is simply configuring the master and slave Jenkins nodes with declarative YAML files and managing them as code. There is no need for a user to learn the Jenkins setup from scratch, rather they can configure Jenkins by a single click. JCaC will help in disabling the initial setup wizard of Jenkins, installing the required plugins and enabling and disabling various security aspects. Such a JCaC setup will help to reproduce and restore the full environment of Jenkins within a few minutes. By converting the manual Jenkins configuration to code, JCaC helps to configure a fully working Jenkins installation with no user interaction needed.
Why JCaC is preferable to plain Jenkins
Reason 1: Manual setup
Configuring Jenkins is a complex process, as both Jenkins and its plugins require some considerable effort, with dozens of parameters that need to be set in the management web interface. You probably are familiar with seeing the following screens:
Step 1
Setting up the initial setup wizard involves going into /var/lib/jenkins/secrets/initialAdminPassword for extracting the single time password that is generated by Jenkins itself.
Step 2
Installation of some plugins as suggested by Jenkins or one can choose to install only selected plugins. Jenkins provides hundreds of useful plugins that eliminate the need to create custom scripting to solve common problems with minimal effort.
Step 3
Creating the first user of the Jenkins dashboard.
At first, one might even think of this process as one time step. However, when you have to repeat the process multiple times or try to recreate your perfect configuration, then comes the reality of a tedious task when going through the above sequence again and again. These steps make setting up Jenkins quite a difficult task. And if a user is a newbie to Jenkins then the user has to start learning it from scratch. But with JCaC a user can simply create the entire configuration in one go.
Configuring Jenkins as code allows you to define the entire configuration as a simple human-readable syntax. Without manual steps, this configuration can be verified and applied to the Jenkins wizard in a fully reproducible manner. Using JCaC makes installing the new Jenkins wizard a simple event.
Reason 2: Master slave configuration
Sometimes a single Jenkins server is not enough to meet certain requirements as different testing environments might be required or a Jenkins server might not handle the load for a larger project. Therefore Master-Slave architecture is set up to meet the actual workload of building projects that are delegated to multiple “slave” nodes, which allows running numerous projects and their jobs. Having multiple slaves allows you to work more quickly without wasting time on waiting for build executors.
In JCaC slaves are automatically connected to the master as per the requirements and unlike old-school Jenkins we don’t need to manually add slave nodes one by one.
Tools used for running JCaC on AWS
Packer is lightweight, portable, and command-line driven. Packer is used to automate the creation of any type of machine images and is easy to use.
Advantages of Packer:
- Pre-baked images
- Superfast infrastructure deployment
- Multi-provider portability
- Improved stability
- Greater testability
Packer configuration templates are written in JSON format. A template has the following three main parts:
- Variables: For defining custom variables
- Builders: Responsible for creating a machine and turning that machine into an image.
- Provisioners: Used for integrating shell scripts, ansible playbooks or chef cookbooks for configuring required applications in the AMI.
Packer only builds images. It does not attempt to manage them in any way. After they’re built, it is up to you to launch or destroy them as you see fit.
2. Amazon Elastic Cloud Compute (EC2) service
Amazon EC2 is a web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers. You can launch an instance from an existing AMI (Amazon Machine Image), customize the instance, and then save this updated configuration as a custom AMI. Instances launched from this new custom AMI include the customizations you made when you created the AMI.
Custom AMI can be also created with the help of Packer and stored as AMIs under Owned by me section. This can be used to launch the instance and Jenkins is wholly setup without any complicated process.
3. Auxiliary AWS Services
- Route 53
- Elastic Load Balancer
- Virtual Private Cloud
- Auto-scaling
- Cloud Watch
For basic knowledge of the above mentioned services you can refer to the official AWS documentation https://docs.aws.amazon.com/index.html?nc2=h_mo
4. Terraform
An Infrastructure-as-Code tool is used for creating and modifying infrastructure safely and efficiently in a declarative manner. It generates an execution plan describing the steps it will execute to reach the desired state, and then executes them to build the specified infrastructure.
Design Architecture
This is the basic architecture of JCaC implemented in AWS using Terraform. We have incorporated the auxiliary AWS services in our solution to leverage the following capabilities:
- Route 53: Used for provisioning domain names for our Jenkins services running on AWS. It helps in public DNS resolution for our solution. Route 53 connects global DNS queries to infrastructure running in AWS, like Elastic Load Balancers, and is used to direct traffic to infrastructure from outside of AWS.
- Elastic Load Balancer(ELB): Used to balance the load on Jenkins master. This helps in preventing the master to go down due to excess load by automatically distributing incoming application traffic to slave nodes. It also adjust capacity according to incoming application and network traffic by communicating with the auto-scaling group.
- Virtual Private Cloud(VPC): We create a public and private subnet each across multiple availability zones in order to achieve HA for our Jenkins. Private subnets run Jenkins Master and Slave servers, while public subnets are for internet facing traffic.
- Auto-scaling: We create an Auto-scaling group which creates a collection of EC2 instances running Jenkins slave servers, and a launch configuration (which is an instance configuration template) that an auto-scaling group uses to launch EC2 instances. This helps in ensuring that the correct number of EC2 instances stay available to handle the load for application.
- Cloud Watch: We create two Cloudwatch alarms for monitoring — one for High CPU Utilization and the other for Low CPU Utilization. For each of the alarms we have two policies named scale-out and scale-in. The scale-out policy scales up by a single instance as soon as the CPU utilization averages 80% and the scale-in scales down by single instance as soon as the CPU utilization averages 20%.
Adopting JCaC
Packer is used for developing master and slave Jenkins AMIs. AMIs contain all the required configuration needed by the master and slave Jenkins node. The configurations are provisioned in the respective AMI using shell script provisioners specified using the Packer templates in JSON format.
Template used for creating AMI:
We’ve used AWS amazon-ebs builder to create a builder. As we want to build the AMI that will be available in the AMI image section.
We’ve used the JSON Packer template to includes basic security files which help in the elimination of initial setup steps and installation of plugins.
- Disabling Initial Setup Wizard
The initial setup wizard is disabled by updating the ‘JENKINS_JAVA_OPTION’ attribute by passing “-Djenkins.install.runSetupWizard=false ” in Jenkins configuration file. The benefit of doing this step is, there is no need for a user to search for the path of the file where the initialAdminPassword is present.
2. Installing Plugins
The plugins installation is automated using a shell script named install-plugins.sh which picks up plugins.txt (containing list of suggested plugins). When you install plugins manually you need to either select the suggested plugins download or manually select the plugins which we want to install. This process is quite tedious. So, while automating this part we simply write down all the required plugins in a single plugins.txt and then run a shell script to install the plugins and their dependencies.
3. Groovy files for security
- basic-security.groovy: This groovy file focuses on creating an Admin Jenkins user i.e. default user. While configuring Jenkins manually we have to do a lot of effort in creating an admin user but with groovy, we can simply pass the username and password for the admin user we want to create.
- disable-cli.groovy: This groovy help in disabling the access to Jenkins CLI for every users other than Admin user and you can check whether the JenkinsCLI has been disabled or not. You can create another user and run the JenkinsCLI commands to check whether they are working or not. If CLI has been disabled properly then you’ll find an exception thrown after running the CLI commands.
- csrf-protection.groovy: Enable csrf protection which helps in preventing Cross-Site Request Forgery exploits. Cross-site request forgery is a class of attack that forces an end user to execute unwanted actions on Jenkins. Because of the way this attack works, even Jenkins that is running inside a corporate firewall is vulnerable. So in order to prevent this class of attack enable csrf-protection.
- disable-jnlp.groovy: Jenkins users use TCP port to communicate with agents launched via JNLP protocol. This file is used to disable jnlp protocols for security purposes.
- jenkins.install.UpgradeWizard.state: This file states the version of Jenkins.
- node-agent.groovy: This groovy connects the master Jenkins with the agents.
Setup
Building the Infrastructure
Step 1: Build the AMIs for Jenkins Master and Jenkins Slaves.
Jenkins Master AMI includes basic groovy security files and installation of plugin.txt file using shell script. Run the following commands from the Packer templates root directory.
sudo packer validate
sudo packer build “ami.json”We end up creating 2 AMIs:
AMI 1: Jenkins Master
Amazon Linux image with Jenkins server including Java, Telegraf, and Git.
AMI 2: Jenkins Slave
Amazon Linux image with Jenkins server includes Docker, Java, and Git.
Step 2: Running Terraform scripts from the Terraform root directories for master and slave for setup of the infrastructure of JCaC.
sudo terraform init
sudo terraform plan -var-file=”jenkins-using-packer.tfvars”
sudo terraform apply -var-file=”jenkins-packer.tfvars”Master Jenkins automation
On running terraform script in the Dev directory an EC2 instance attached with ELB is launched in a VPC with the inbound traffic ports opened in the Security Group for the internet connectivity.
Slave Jenkins automation
On running terraform script in Dev directory, minimum two EC2 instances in the auto-scaling group are launched in a VPC with the inbound traffic ports opened in the Security Group for the internet connectivity. Cloudwatch alarms are set for scale-in and scale-out.
After running above steps, Jenkins master and slaves would be set up as a part of the infrastructure and anyone just needs to hit the URL of Elastic Load Balancer at its default port to access Jenkins and sign up using the Admin credentials and Jenkins is up and ready to build jobs.
Finally, we’ve automated everything with JCaC and Jenkins is ready for use.
Conclusion
The main objective behind automated Jenkins was to run a single command for its setup. The result is a fully configured Jenkins setup that we can multiply to as many instances as we need. We’ve automated the setup of an administrator user, and the installation of Jenkins plugins, and the initial setup wizard is disabled by updating the Jenkins configuration file. The main aim is achieved easily with the help of custom AMIs hosted on the AWS platform which requires no additional software installation and some of the free-tier features make it accessible for most people. Having JCaC, we can now spawn it as many times as possible, on different machines with the same result, with little to no tiresome manual work through the GUI.
