Installing Jenkins CI/CD Tool on an Amazon EC2 Instance Using Terraform

In this project, I deployed a Jenkins web server on an EC2 instance using Terraform infrastructure as Code (IaC). I also deployed an S3 bucket in AWS using Terraform to store the Jenkins artifacts.

Brief Overview of Terraform

Terraform is an Infrastructure as Code (IaC) tool that allows you to define and provision cloud infrastructure that you can version, reuse, and share. Therefore, it facilitates consistency and collaboration across different environments. An important benefit of Terraform is the automation of infrastructure provisioning and management. Terraform is written in HashiCorp Configuration Language (HCL), a human-readable high-level configuration language.

So what sets Terraform apart from other IaC tools like AWS CloudFormation? It’s its ability to provision and manage various types of infrastructure, e.g., servers, databases, etc., across multiple cloud providers and on-premise environments. Terraform has thousands of providers to manage the resources deployed, and they can be found in the Terraform Registry.

The Terraform Workflow

It is important to understand the consistent workflow of Terraform to know how to use it to deploy and manage resources. Although Terraform has 3 core workflows: Write, Plan, and Apply, there are some additional steps to be taken into account when you would like to deploy a Terraform infrastructure. The below consists of the Terraform Workflow:

Scope: This is the process of identifying the resources needed for your infrastructure.

Write: Author the configuration to define your infrastructure; develop your IaC.

Initialize: Install the required Terraform providers needed to build your infrastructure.

Plan: Preview the changes Terraform will make to match your configuration before execution.

Apply: Make the changes to your infrastructure by deploying your reproducible infrastructure.

Basic Terraform Commands

Below are some important Terraform commands that are important to know and would be used in this project:

terraform init : This command initializes a new Terraform working directory containing Terraform configuration files. It downloads and installs the necessary provider plugin. This command is typically the first to be run after writing a new Terraform configuration.

terraform plan : This command displays the actions Terraform will take to achieve the desired state described in the configuration files without actually applying the changes. It checks whether the changes proposed by Terraform match your expectations before applying those changes.

terraform apply : The Terraform apply command executes the actions proposed in the Terraform plan. Each change required to reach the desired state of the configuration is applied here.

terraform destroy : As the name implies, this command destroys all remote resources or infrastructure managed by a Terraform configuration.

terraform fmt : This command ensures that your configuration files are easy to read and follow Terraform’s standard formatting practices by formatting your configuration files in a canonical format and style.

terraform validate : Terraform validate confirms the syntax and configuration of the Terraform files in the directory. It checks whether the configuration is syntactically valid and internally consistent.

Use Case/Challenge

Your DevOps team would like to start using Jenkins as their CI/CD tool to create pipelines for DevOps projects. They need you to create the Jenkins server using Terraform so that it can be used in other environments and so that changes to the environment are better tracked.

Solution

To solve the challenge above, I would be going through how to:

• Use the AWS provider from Terraform
• Deploy an EC2 instance with a user-data script to install and run Jenkins in your default VPC
• Create a security group rule for the Jenkins server that allows SSH, HTTP, and traffic on port 8080 from my IP
• Create a private S3 bucket to store the Jenkins artifacts

All done using Terraform.

Prerequisites

1. An AWS account. You can sign up for free here.
2. Knowledge and know-how to deploy an EC2 instance in the AWS console.
3. A source code editor to build and edit your infrastructure code. I used Visual Studio Code.
4. The Terraform extension installed in your VSCode.
5. Terraform installed in the CLI of your local computer. You can find the installation tutorial here
6. Knowledge of basic terraform commands

All the Terraform configuration files I created for this project can be found in a Github repo in my GitHub account here

learning-terraform/terraform-aws at main · nifss-sofowoke/learning-terraform

Step 1 (Optional): Deploy & Bootstrap an EC2 instance with a script to install Java and Jenkins

• Although this step is optional, I recommend doing this project in the AWS console as the first workflow for this project. This would help us identify the resources needed to input in our Terraform configuration file when we want to deploy the instance using IaC.
• I carried out this project in my AWS console, verified the Jenkins server ran, and noted the resources I used.

used an ubuntu instance

created a new security group

inbound security group rules

verified the user data script worked and Jenkins was running

Resources noted that would be needed for the Terraform configuration file:

• Ubuntu instance ami-id
• Instance type
• User data script to install and run Jenkins
• Default VPC ID
• Key pair used
• New security group allowing traffic on port 22 & 8080 from my IP, and web traffic

Step 2: Create a New Folder with the Required Configuration Files

• In VSCode in a new terminal window, create a new folder for the project.

mkdir <directory-name>

• Navigate to the folder and create the following 4 Terraform configuration files: providers.tf, main.tf, variables.tf, & outputs.tf

providers. tf: this file is used to specify the particular Terraform provider that would be used e.g., Google Cloud, AWS, etc, and their configurations. Specifying providers in this file enables us to interact with the resources supported by that cloud provider.

main.tf: this is the main Terraform file where all the resources to be deployed, e.g EC2 instance, VPC, S3 bucket, etc would be defined

variables.tf: this file is used to declare input variables, which allow aspects of the main configuration file to be customized or serve as parameters. Doing this simplifies and increases the usability of your Terraform configuration.

outputs.tf: This file specifies the output values that you want Terraform to return after the infrastructure is created or updated. This could include things like instance ID, public IP, etc.

Step 3: Configure AWS Credentials

• In the terminal, using the AWS CLI, run the aws configure command. This would prompt us to enter the access key, secret access key, and default region.
• Another way to do this would be to set the credentials in your terminal session by exporting them as environment variables in the config file of your shell

export AWS_ACCESS_KEY_ID="youraccesskey"
export AWS_SECRET_ACCESS_KEY="yoursecretkey"
export AWS_DEFAULT_REGION="us-east-1"

Step 4: Install & Configure Terraform AWS Provider

• Navigate to the providers.tf file and add the required providers block below:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.55.0"
    }
  }
}

The latest version of a Terraform provider can be found in the Terraform Registry

• Configure the AWS provider by adding the code block below to the file:

provider "aws" {
  region = "us-east-1"
}

This informs Terraform that it will deploy resources/services in the ‘us-east-1’ region within AWS.

Altogether, the code block for the providers.tf file should look like this:

providers.tf file

Step 5: Initialize the providers.tf file

• Using the command below in the terminal,

terraform init

initialize the providers.tf file. Initializing the providers.tf file will cause Terraform to take note of the configuration and install the required provider (AWS).

• Verify that the provider has been installed by running the terraform version command

Step 6: Build the main.tf file with the required resources

In the main.tf file, add the resources that you would like Terraform to deploy in AWS using resource blocks. I used the Terraform Registry documentation on AWS services to build out my code.

EC2 Instance resource block

I used the code block below to deploy an Ubuntu EC2 instance with Jenkins installed and running on it. This resource block contains the required details needed to deploy the instance like the instance type, ami-id, security group ID, user data script, and so on.

# Create an EC2 Instance

resource "aws_instance" "jenkins-server" {
  ami           = "ami-04b70fa74e45c3917"
  instance_type = "t2.micro"
  key_name      = "Nife-LUIT_KEYS"
  vpc_security_group_ids = [aws_security_group.jenkins-SG]
  associate_public_ip_address = true

  tags = {
    Name = "jenkins-server"
  }

  user_data = <<-EOF
    #!/bin/bash

    #Update the package index
    apt-get update

    # Install the default JDK (or specify a version if needed)
    apt-get install -y default-jdk

    # Set the JAVA_HOME environment variable
    JAVA_HOME=$(sudo update-alternatives --config java | grep -oP '(?<=\s)[^/]+(/[^/]+)+(?=/bin/java)')
    echo "export JAVA_HOME=$JAVA_HOME" >> /etc/profile.d/jdk.sh
    echo "export PATH=\$JAVA_HOME/bin:\$PATH" >> /etc/profile.d/jdk.sh

    # Source the profile to load the environment variables
    source /etc/profile.d/jdk.sh

    # Download the Jenkins GPG key and save it to /usr/share/keyrings/jenkins-keyring.asc
    sudo wget -O /usr/share/keyrings/jenkins-keyring.asc https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

    # Add the Jenkins repository to the system's APT sources list
    echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/" | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null

    # Update the local package index to include the latest information about available packages
    apt-get update

    # Install Jenkins using the apt-get package manager
    apt-get install -y jenkins

    # Start Jenkins at boot
    systemctl enable jenkins

    EOF

}

Security Group Resource

Using the resource block below, I created the security group ‘jenkins-SG’ with the necessary security group rules for the Jenkins server that allow:

• HTTP traffic on port 80 from my IP address.
• SSH traffic on port 22 from my IP address.
• traffic on port 8080 from my IP address.
• all outbound traffic.

# Create Instance Security Group
resource "aws_security_group" "jenkins-SG" {
  name        = "jenkins-SG"
  description = "Allow SSH and Web traffic from my IP and allow all outbound traffic"
  vpc_id      = "vpc-000d6a8c3f9ac9687"

  tags = {
    Name = "jenkins-SG"
  }

    # Create Ingress Rule to allow Web Traffic
    ingress {
        cidr_blocks = ["myIP"]
        from_port   = 80
        to_port     = 80
        protocol       = "tcp"
        }
    # Create Ingress Rule to allow SSH from my IP
    ingress {
        cidr_blocks = ["myIP"]
        from_port   = 22
        to_port     = 22
        protocol       = "tcp"
        }

    ingress {
    # Create Ingress Rule to allow Port 8080 from my IP
        cidr_blocks = ["myIP"]
        from_port   = 8080
        to_port     = 8080
        protocol       = "tcp"
        }

    egress {
    # Create Egress Rule
        cidr_blocks = ["0.0.0.0/0"]
        from_port   = 0
        to_port     = 0
        protocol       = "-1"
        }

}

S3 Bucket Resource

An S3 bucket named ‘jenkins-artifact-bucket-ns’ was defined and deployed using the resource block below

# Create a private S3 bucket for Jenkins' artifact (remember: s3 bucket names have to be unique)
resource "aws_s3_bucket" "jenkins-artifact-bucket-ns" {
  bucket = "jenkins-artifact-bucket-ns"

  tags = {
    Name = "jenkins-artifact-bucket-ns"
  }
}

S3 Bucket Ownership Control Resource

The resource block below was used to configure ownership control for the S3 bucket with a rule that objects are writable to the bucket by the owner only.

# Create S3 bucket ownership control
resource "aws_s3_bucket_ownership_controls" "jenkins_bucket_acl_ownership" {
  bucket = aws_s3_bucket.jenkins-artifact-bucket-ns.id
  rule {
    object_ownership = "ObjectWriter"
  }
}

S3 Bucket ACL Resource

This resource block sets the Access Control List for the bucket as ‘private’ ensuring that the public does not have access and only the bucket owner does.

# Create S3 bucket acl
resource "aws_s3_bucket_acl" "jenkins_bucket_acl" {
  bucket = aws_s3_bucket.jenkins-artifact-bucket-ns.id
  acl    = "private"
}

Putting all the resource blocks together, the main.tf file looks like this:

Step 7: Add Input Variables to the Main.tf file

To manage the configurations in the main.tf file and to make it easier to use and maintain, I added variables using the variable.tf file to declare the variables. Using the variables.tf file prevents hardcoding sensitive data, & values like the VPC ID, and instance ami-id, which could cause errors. It also allows us to change values without modifying the main configuration files directly.

The code template for declaring a variable in the variable.tf file is shown below:

variable [variable name] {
  type    = [variable type]
  description = [description string]
  default = [default value of the variable]
}

The variables declared in the variables.tf file are shown below:

variable.tf

After declaring the variables, ami aws-region vpc_id cidr_blocks key_name and instance_type , update the main.tf file to reflect the changes by adding ‘var.’ to the name of the variable as seen below.

Step 8: Define Output Values

Used the output.tf file to define the output values that would be needed after deploying the infrastructure. These are:

• instance public IP URL, which points to port 8080 and would be used to access the Jenkins server
• instance id
• instance public DNS

Step 9: Format & Validate Configuration Files

• In the CLI, use the terraform fmt command to ensure that all files in the directory are formatted properly.
• After formatting the files, confirmed that the syntax of the code was correct and was free of syntax errors using the terraform validate command.

Step 10: Terraform Plan

Run theterraform plan command to see the proposed changes Terraform would make to deploy the services in the configuration file.

terraform plan

As seen above, Terraform would create 5 resources, change 0, and destroy 0.

Step 11: Terraform Apply

Use the terraform apply command to execute the changes proposed in the plan.

Tip: you can use the command terraform apply -auto-approve to automatically apply the changes without being prompted to answer yes/no.

terraform apply

Step 12: Verify Resources were Created

• In the AWS console, it was confirmed that Terraform deployed the services it was supposed to create.

Jenkins instance was deployed

the security group was created

private s3 bucket was created

• SSHed into the instance to confirm Jenkins was installed and is running. Used the command below:

systemctl status jenkins

• Used the instance_public_ip_url outputted in the terminal after running the terraform apply command to access the Jenkins server.

Jenkins server running!

Step 13: Clean Up

Used the terraform destroy command to remove all resources and services deployed to prevent incurring unwanted charges.

Conclusion

In this project, I:

• briefly explained Terraform IaC
• explained the Terraform workflow
• listed and explained some basic Terraform commands
• briefly explained the Terraform configuration files
• explained step-by-step how to use the Terraform workflow and configuration files to deploy an EC2 instance with Jenkins installed
• created a private S3 bucket to store Jenkins’ artifacts

Thanks for reading! I hope you found this project valuable.

Feel free to connect with me on LinkedIn or leave any constructive feedback you have in the comments.