Automating DevOps: Building a Jenkins CI/CD Server with Terraform for Scalable Environments

Scenario: Your team would like to start using Jenkins as their CI/CD tool to create pipelines for DevOps projects. They need you to create the Jenkins server using Terraform so that it can be used in other environments and so that changes to the environment are better tracked.

What is Jenkins exactly? Jenkins is an open-source automation server that facilitates the Continuous Integration (CI) and Continuous Delivery (CD) processes in software development. A Jenkins CI/CD server streamlines the development and release process, leading to faster feedback, reduced manual intervention, and higher software quality. It’s a crucial tool in modern software development workflows that emphasizes automation and collaboration.

Pre-requisites:

• AWS and Cloud9 Environment Knowledge
• Terraform Documentation
• Link to my GitHub with Project Code

Objectives:

1. Deploy one EC2 instance in your default VPC.
2. Bootstrap the EC2 instance with a script that will install and start Jenkins.
3. Create and assign a security group to the Jenkins security group that allows traffic on port 22 from your IP and allows traffic from port 8080.
4. Create a S3 bucket for your Jenkins artifacts that is not open to the public.
5. Verify that you can reach your Jenkins install via port 8080 in your browser.

Advanced:

1. Add a variables.tf file and make sure nothing is hardcoded in your main.tf.
2. Create separate file for your providers.tf.

Step 1: Set Up Your Cloud9 Environment

We’ll begin by logging into our Cloud9 environment in AWS. If you don’t have a Cloud9 environment, directions on how to create one are linked here.

We need to create a folder in our repository that will house all of the files needed for this project.

In the terminal, change directories into the folder you just created.

cd <newfolder>

Next, we need to create several files inside this folder. This command will create all of the files at once.

touch main.tf variables.tf providers.tf script.sh

Each file serves a different purpose:

• main.tf: This is the primary configuration file where you define the resources, settings, and configurations you want Terraform to create and manage within your cloud infrastructure.
• providers.tf: Defines the configuration for different cloud providers or backends that your infrastructure uses, such as AWS.
• script.sh: Installs and configures software packages or dependencies needed for your Jenkins server.
• variables.tf: Defines input variables for your infrastructure configuration, making it more flexible, reusable, and easier to manage.

Step 2: Create your Files

Providers.tf: Access the official Terraform documentation for the AWS provider here, then copy and paste the Example Usage code into the providers.tf file. You can configure this code as needed if you would like to use a different version or region.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "us-east-1"
}

Variables.tf: This file is essential in ensuring that nothing is hardcoded in our main.tf file. Using the Terraform documentation linked here, we’ll build out each block to customize our instance in the variables.tf file.

variable "instance_name" {
  description = "Name of the EC2 instance"
  type        = string
  default     = "ExampleAppServerInstance"
}
variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t2.micro"
}
variable "instance_ami" {
  description = "AMI of EC2 instance"
  type        = string
  default     = "Unbuntu AMI id"
}
variable "vpc_id" {
  description = "Id of your VPC"
  type        = string
  default     = "Input VPC id"
}
variable "security_group_name" {
  description = "Security group name"
  type        = string
  default     = "Input security group name"
}

Main.tf: This file holds the core of your infrastructure provisioning instructions. It is best practice not to hardcode the main.tf file as we want our code to be more modular, maintainable, and adaptable. I placed links to the Terraform documentation so that you can utilize it to create each resource block.

Here are the four resource blocks we will be creating:

• “aws_instance” “ubuntu”: Create an EC2 instance
• “aws_security_group” “Jenkins_SG”: Create a security group for Jenkins
• “aws_s3_bucket” “jenkins_bucket”: Create an S3 bucket for Jenkins
• “random_id” “randomness”: Generate a random id

resource "aws_instance" "ubuntu" {
  instance_type          = var.instance_type
  ami                    = var.instance_ami
  vpc_security_group_ids = [aws_security_group.Jenkins_SG.id]
  tags = {
    Name = var.instance_name
  }

  user_data = file("script.sh")
}

resource "aws_security_group" "Jenkins_SG" {
  name        = var.security_group_name
  description = "Inbound rules for security group"
  vpc_id      = var.vpc_id

  ingress {
    description = "ssh"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "incoming 8080"
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "incoming 443"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    description = "outbound rules"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  resource "aws_s3_bucket" "jenkins_bucket" {
    bucket = "jenkins_bucket-${random_id.randomness.hex}"

    tags = {
      Name = "jenkins_bucket"
    }
  }

resource "random_id" "randomness" {
  byte_length = 10
}

Script.sh: This file contains the script that fetches and installs Jenkins, an automation server for CI/CD, on the EC2 instance. This script is executed as part of the instance launch process, making it a fundamental tool for initializing resources consistently and reliably. Copy the below script and paste it in the script.tf file.

#!/bin/bash
sudo apt-get update -y
sudo apt-get install openjdk-11-jre -y
sudo curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee   /usr/share/keyrings/jenkins-keyring.asc > /dev/null
sudo echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install fontconfig openjdk-11-jre
sudo apt-get install jenkins -y 
sudo apt-get systemctl start jenkins
sudo apt-get systemctl enable jenkins

Step 3: Run Commands in the CLI

Now that we’ve created all of the files that are essential to generating our server, we have to run a few commands to execute it.

The first code you will need to run is terraform fmt. The terraform fmt command is used in Terraform to automatically format your Terraform configuration files according to a consistent style guide. It helps ensure that your code is well-organized, readable, and follows a standardized format.

terraform fmt

Next, you will run terraform init. The terraform init command is used in Terraform to initialize a working directory that contains Terraform configuration files. Always remember to run this command when you start working on a new Terraform project or configuration, or if you have made changes to the configuration that involve new providers, modules, or backend configurations.

terraform init

Now, we’ll utilize terraform plan to create a blueprint. The terraform plan command is used in Terraform to preview the changes that Terraform will make to your infrastructure if you apply the current configuration. It gives you an overview of the actions Terraform will take without actually making any changes to your environment.

terraform plan

Lastly, we need to implement this plan by running terraform apply. The terraform apply command in Terraform is used to apply the changes defined in your configuration to your infrastructure. Another option is to use terraform apply -auto-approve. The -auto-approve flag instructs Terraform to automatically apply the changes without requiring manual confirmation. This is particularly useful when you’re certain about the changes being made and don’t need to review them.

terraform apply
terraform apply -auto-approve

Step 4: Verify Resource Creation

Let’s head over to the AWS console to verify that all of our resources have been created properly. In the EC2 console, we can see that our Jenkins_EC2 instance has been created.

Next, scroll down and click on Security Groups to see that our Jenkins_SG security group is created.

Head over to the S3 console to verify the creation of our Jenkins-bucket. Notice how the random id generator that we used in our code created a unique bucket name.

Finally, we need to verify that we can reach our Jenkins webpage. Copy the public IPv4 address of your EC2 instance and paste it into a web browser followed by the port number, :8080.

<PublicIPAddress>:8080

Success! We have reached the interface and control center for managing Jenkins!

Step 6: Burn it Down

The very last step to wrap up this project is to burn it to the ground. The terraform destroy command in Terraform is used to tear down and delete the resources that were created by your Terraform configuration. Like terraform apply, you can also use the -auto-approve flag to automatically apply these changes. Although there are benefits of using terraform destroy, such as preventing unnecessary costs, be sure to exercise caution and confirm that you are deleting the correct resources.

terraform destroy
terraform destroy -auto-approve

Congratulations, you have successfully created and destroyed a Jenkins CI/CD server using Terraform!

Be sure to drop me a clap, comment, and follow me as I continue my journey through the cloud.