Launching a Jenkins Server with Terraform

A Beginner’s Guide to Deploying a Jenkins Server with Terraform

Edited Adobe Free Stock Image & Logos

Welcome! Today, I will be serving up a beginner’s guide to deploying your first Jenkins server with Terraform.

I am super excited to be learning the basics of Terraform and to share this project with you. As always I will try to break it down in an easy to follow along format. Remember if you run into errors, it will be ok.

Learning how to troubleshoot will only make you a stronger DevOp engineer.

A little background //

Terraform is an infrastructure as code tool that allows DevOp Engineers to build, change, and version cloud and on-prem resources safely and efficiently.

CI/CD Pipeline is the path of continuous integration and continuous delivery/deployment of software

Jenkins is a world wide open source automation server enabling developers to reliably build, test, and deploy their software.

Scenario //

Your team would like to start using Jenkins as their CI/CD tool to create pipelines for DevOps projects. You are assigned to create the Jenkins server using Terraform, allowing it to be used in other environments and for changes to be tracked.

Objective //

• Deploy 1 EC2 Instances in your Default VPC
• Bootstrap the EC2 instance with a script that will install and start Jenkins
• Create and assign a Security Group to the Jenkins Security Group that allows traffic on port 22 from your IP and allows traffic from port 8080
• Create a Private S3 bucket for your Jenkins Artifacts
• Verify that you can reach your Jenkins install via port 8080 in your browser. Be sure to include a screenshot of the Jenkins login screen in your documentation.

To follow along with this project you will need //

• Access to AWS
• An Configured AWS Cloud9 with AWS CLI
• An Optional GitHub Account
• Attention to Details

If you have followed me on previous projects, you know how visual I am. I find that providing imagery helps with guiding you through a project as well as aid in understanding the process. I am a big fan of utilizing documentation, it helps prevent errors, and my first stop for troubleshooting. I have attached the documentation I referenced while working on this project under the section, Helpful Resources.

You ready? If so, let’s begin by setting up our AWS Cloud9 Environment.

Creating an AWS Cloud9 Environment //

Note: For this project I will be working with AWS Cloud9 Environment. One of the benefits of using Cloud9, is Terraform is pre-installed. I have provided the set-up incase you are unfamiliar or need a refresher. You may also utilize an IDE (integrated development environment) of your choice.

• Log into your AWS Console
• Navigate to Cloud9 via the search

Note: I am using AWS Region US East 1 (N. Virginia), it is the largest Region in the US, and just happens to be closest to me. If you live on the West Coast and run into any issues you can try always terminate/delete and update Region and try the project again.

• Select Create environment

• Create a Name
• Select New EC2
• Set Instance Type as t2.micro

• Select AWS System Manager
• Select Create

Note: Creation can take several minutes

Successful Creation of Cloud9 Environment

• Navigate to EC2 Dashboard
  An EC2 was created, and currently is running Cloud9 Environment

Note: We left a default setting that after 30 minutes of inactivity, your EC2 instance will stop to prevent unnecessary charges. It is best practice to stop your EC2 anytime you plan on leaving your project.

• Navigate back to Cloud9 console
• Select Environment
• Select Open in Cloud

Setting up Cloud9 for Success//

Once your Cloud9 Environment opens we can set our foundations for success.

• Configure AWS Credentials
  Now, that we have our Cloud9 Environment created, we will need to configure it with our AWS Key Credentials. This will allow for the AWS provider to interact with the resources we will be creating today. In the past, I have demonstrated with credentials inside of code. However, it is best practice to configure using the command line.
• Run the following commands separately with your information from the Cloud9 Terminal

export AWS_ACCESS_KEY_ID=

export AWS_SECRET_ACCESS_KEY=

export AWS_DEFAULT_REGION=

Note: If you misplaced or need to create new you can following the documentation linked here.

Verify Terraform //

As I mentioned a benefit to using AWS Cloud9, is Terraform is already installed. From our Cloud9 Terminal we can verify.

• Verify Terraform

terraform version

Sweet!

The next step is optional, but I highly recommend.

Add Optional GitHub Repo & Creating New Directory //

If you are in the practice to pushing any code you write to GitHub, you might want to add your GitHub.

• Select Source Control from left hand menu
• Select Clone Repo

• Add Clone GitHub Repo link
• Create new branch from Source Control Tab
  It is best practice to create a new branch to prevent from directly merging files without verification/approval.
• Change Directory

cd <Clone Repo>

• Create new Directory example:wk20Terraform

mkdir <NAME YOUR DIRECTORY>

Environment Example

Awesome, we are set up, organized and ready to move with working with Terraform!

Following the Terraform Workflow //

Remember, I said that Terraform is an infrastructure as code tool? Terraform creates and manages resources and other services through their application programming interfaces (APIs). Providers are what enables Terraform to work with services with an accessible API. To achieve this Terraform follows a core workflow. (You can view in-depth in the provided Documentation under the Helpful Resources Section.)

The core Terraform workflow consists of three stages:

Write: You define resources.

Plan: Terraform creates an execution plan describing the infrastructure it will create, update, or destroy based on your configuration and existing infrastructure.

Apply: On approval command, Terraform performs the proposed operations in the correct order, respecting any resource dependencies.

We will begin working with Terraform by creating a main.tf file. This will be a single configuration file, you don’t have to name it main, but it has to have the .tf extension to be recognized by Terraform.

• Create File → New Text File → Save As → main.tf → Save

Note: Make sure you are saving it in your Directory you created earlier

Tips for building a main.tf File //

• In this portion of the project we will be our AWS default VPC and associated default subnets. If you need to recreate your default you can create following this AWS documentation.
• Test your code as you go. Every few resources you create run your Terraform Workflow to make sure everything works, this will allow for easier troubleshooting to resolve any potential errors.

Building our main.tf File //

Note: All code is built from documentation noted in the Helpful Resource Section. It is also important to know, Terraform files are written in HCL (Hashicorp Configuration Language). Again, it is completely normal not know how to write it all out, we are still learning. This is why I highly encourage reviewing all attached documentation.

Adobe Free Stock Image

Ready? Let’s get started writing our main.tf file (Your file after saving should have remained open at the top portion of your Cloud9 Environment. If not, File → Open → main.tf)

• Configure provider

#Configure the AWS Provider
provider "aws" {
  region = "us-east-1"

• Choose AMI template

Navigate to AWS EC2 → Select Images from left hand menu→ Select AMI Catalog →Scroll Down till Ubuntu Free tier eligible→ Copy AMI

Returning back to our main.tf we will continue to build our code.

• Define resource of a single EC2 Instance
  -Name
  -Specify AMI
  -Specify Instance Type

#Create EC2 Instance
resource "aws_instance" "instance1" {
  ami                    = "ENTER AMI ID"
  instance_type          = "t2.micro"
  tags = {
    Name = "wk20jenkins_instance"
  }

• Bootstrap Jenkins to EC2 by defining in user_data portion of our EC2

#Bootstrap Jenkins 
  user_data = <<-EOF
  #!/bin/bash
  sudo apt update -y
  sudo apt install openjdk-11-jre -y
  curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
    /usr/share/keyrings/jenkins-keyring.asc > /dev/null
  echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
    https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
    /etc/apt/sources.list.d/jenkins.list > /dev/null
  sudo apt-get update -y
  sudo apt-get install jenkins -y
  sudo systemctl enable jenkins 
  sudo systemtl start jenkins
  EOF

  user_data_replace_on_change = true
}

Our code includes the install of Java, as Jenkins requires Java in order to run, it is important to note that certain distributions don’t include this by default.

Here is an example of our main.tf code so far:

Note: After I took the screenshot I realized I left off all that approval -y. They are added into the code block. (<<-EOF and EOF are Terraform’s heredoc syntax and allows you to create multi-line strings)

• From Terminal we will check our work so far

cd <into directory containing main.tf>

terraform init

terraform init — initializes cwd (current working directory) containing Terraform configuration files. Always the first command that should be run after writing a new Terraform configuration. Bonus it is safe to run this command multiple times.

• Validate
  terraform validate — verifies whether a configuration is syntactically valid and internally consistent, regardless of any provided variables or existing state.

terraform validate

Success! Configuration Syntax is valid

Awesome job so far!! Feel free to get up, grab a cup of joe, or just stretch if you have been following along.

Adobe Free Stock

It can seem like a lot of information being thrown at you all at once, but with practice and hands on experience it will start to click.(Remember if you stop at any point, and pause the project; STOP your EC2)

When you are ready, let’s continue on building our main.tf by creating our Security Group.

Creating a Security Group //

You may be familiar with setting up a VPC and Security Groups using the AWS console. However, our main goal is to write one document that does it all. Our objective is to create a Security Group that allows the following.

Ingress (inbound ports):

• Open Port 22
  This allows SSH from our local computer’s IP address
• Open Port 8080
  We will receive traffic from any IP address
• Open Port 443
  Allowing HTTP traffic

Egress (outbound ports): set to 0

Note:
Since we are building one document, we want to ensure that we do not hardcore any personal information. Especially, if you plan on pushing code to GitHub. You can use your specific IP for Port 22, but for this walk-through I will be using CIDR block 0.0.0.0/0. Using this CIDR block will allow traffic from any IP address.

#Create security group 
resource "aws_security_group" "wk20sg_jenkins" {
  name        = "wk20sg_jenkins"
  description = "Open ports 22, 8080, and 443"

  #Allow incoming TCP requests on port 22 from any IP
  ingress {
    description = "Incoming SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  #Allow incoming TCP requests on port 8080 from any IP
  ingress {
    description = "Incoming 8080"
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  #Allow incoming TCP requests on port 443 from any IP
  ingress {
    description = "Incoming 443"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  #Allow all outbound requests
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "wk20sg_jenkins"
  }
}

• Validate

terraform validate

Good to go! Let’s continue

Our last step with our SG is to assign it to our EC2. We can assign using an expression using the name we gave our security group. Add the following expression in the #Create EC2 Instance portion of our main.tf.

vpc_security_group_ids = [aws_security_group.wk20sg_jenkins.id]

Note: Since our SG does not have an ID assigned to it, we reference the name

Three objectives are now complete! Two more to go! YOU GOT THIS!

Create a Private S3 Bucket //

Continuing to build our main.tf, the next objective is to create a Private S3 Bucket. This will hold all the Jenkin artifacts, which are files generated during execution of a Jenkins pipeline. If you have created a S3 Bucket before, you know the name is globally unique. After some researching, I decided to add the resource random_id from the Terraform documentation. This added to our bucket name will ensure that it is globally unique.

Note: As of April 2023 AWS made all S3 Buckets private by default.

#Create S3 bucket for Jenkins artifacts
resource "aws_s3_bucket" "wk20jenkins-artifacts" {
  bucket = "wk20jenkins-artifacts-${random_id.randomness.hex}"

  tags = {
    Name = "wk20jenkins_artifacts"
  }
}

#Create random number for S3 bucket name
resource "random_id" "randomness" {
  byte_length = 8
}

Add code to our main.tf under our SG section. If you are following along main.tf should be looking something like this:

Whew! That’s a lot of code there isn’t it. Again, validating along the way to prevent errors.

Note: If you close your Terminal you will need to run terraform init before running terraform validate

We have built our main.tf, and now it’s the time to deploy our infrastructure!

Deploy our Infrastructure //

Terraform Workflow: write → plan → apply

• Ensure you are in the CWD that your main.tf file is located
• Run terraform init, if you have opened a new terminal or restarted the project after a pause
• Run terraform validate, to verify syntax once again

Moving on through the workflow, we are in the plan stage. I have becoming in the habit of running terraform fmt to check formatting and correct the spacing to correct HCL format.

terraform fmt

Next, we will execute terraform plan which creates an execution plan listing all the resources that will be created once the Terraform file is applied.

terraform plan

Nervous, I am a little. I feel like that is normal after building a code. Everything looks go so far, let’s run terraform apply. This will create the resource infrastructure as we have defined in our main.tf file.

terraform apply

After a moment or two, your screen will display the actions Terraform will perform, followed by a prompt to enter ‘yes’ to approve.

After a few moments, your output should show Apply complete!

Let’s check everything on our AWS console.

• Navigate to EC2

• Navigate to Security Groups from EC2 left hand menu

Success!

Successful SG Created!

• Navigate to S3

Successful S3 Bucket Created!

Everything looks great on this side, let’s continue!

Verify Jenkins //

• Navigate to EC2 Dashboard
• Open Jenkins EC2
• Select Connect

When we created our EC2 we did not set up a .pem file key so we will not be able to ssh into. However, if you choose the first tab EC2 Instance Connect we will be able to connect through AWS.

• Select EC2 Instance Connect
• Select Connect

A new window tab should open and you are now connected to your Jenkins EC2.

• Verify Jenkins

sudo systemctl status jenkins

ctrl+ c to exit then enter to bring back to command line

• Return back to Cloud9 Environment
• Verify using curl

curl <instance_publicip>:8080

Awesome, we are definitely getting feedback from a html!

• Validate on web browser

http://<instance_publicip>:8080

Woo-Hoo!! Success!!

Adobe Free Stock

Optional GitHub //

Congratulations! If you have made it to this point you have successful built and deployed a Jenkins server with Terraform. Your team is set up for success to utilize this server for their CI/CD pipeline. Don’t forget to push your code to your GitHub account. Need a quick review on basic’s of GitHub, feel free to check out my earlier article. For quick reference here is a link this project:
https://github.com/mel-foster/Terraform/tree/main/wk20Terraform

Clean Up Time //

It’s time to clean up! We want to ensure all EC2s are stopped and terminated as we no longer need. Plus tear down our Security Group & S3 Bucket. The great news is, we can do it all with one command.

terraform destroy

• You can review by scrolling what will be destroyed
  Note: 4 to destroy

Type ‘yes’ press enter to continue

Adobe Free Stock Image

That’s a wrap for today’s project! Terraform can create some long files. There’s actually a way to have streamlined main.tf utilizing a variable.tf file, amongst other options. Check out my follow-up article, where we get a little more advanced. Thank you for joining me today! Be on the look out for another project coming your way soon.

Medium

Helpful Resources //

What is Terraform | Terraform | HashiCorp Developer

The Core Terraform Workflow - Guides | Terraform | HashiCorp Developer

Documentation | Terraform | HashiCorp Developer

Terraform Registry

Linux

Note from the author: This article has an updated title to reflect setting up a Jenkin server for use of a CI/CD pipeline in the future.

Join me on https://www.linkedin.com/in/melissafoster08/ or follow me at https://github.com/mel-foster

All Screen Capture Images ©Melissa (Mel) Foster