Immutable Infrastructure Using Packer, Ansible, and Terraform

Step-by-step journal with best practice and more …

Just in case you are an explorer, here I provide you with files. https://github.com/lightninglife/ImmutableInfrastructure

Traditional mutable server infrastructure vs. Immutable infrastructure

In a traditional mutable server infrastructure, servers are continually updated and modified in place. Engineers and administrators working with this kind of infrastructure can SSH into their servers, upgrade or downgrade packages manually, tweak configuration files on a server-by-server basis, and deploy new code directly onto existing servers. In other words, these servers are mutable; they can be changed after they’re created. Infrastructure comprised of mutable servers can itself be called mutable, traditional, or (disparagingly) artisanal.

An immutable infrastructure is another infrastructure paradigm in which servers are never modified after they’re deployed. If something needs to be updated, fixed, or modified in any way, new servers built from a common image with the appropriate changes are provisioned to replace the old ones. After they’re validated, they’re put into use and the old ones are decommissioned.

The benefits of an immutable infrastructure include more consistency and reliability in your infrastructure and a simpler, more predictable deployment process. It mitigates or entirely prevents issues that are common in mutable infrastructures, like configuration drift and snowflake servers. However, using it efficiently often includes comprehensive deployment automation, fast server provisioning in a cloud computing environment, and solutions for handling stateful or ephemeral data like logs.

Normal Flow

Diagram of Normal Flow

Immutable Flow

Diagram of Immutable Flow

We get started by using terraform to provision our servers and later employing ansible on instances for configuration management. This allows us to add times when provisioning servers. So that the process will not continue until configuration completes. In terms of configuration, we will be using Packer. Packer is an open source tool for creating identical machine images for multiple platforms from a single source configuration. Packer is lightweight, runs on every major operating system, and is highly performant, creating machine images for multiple platforms in parallel. Packer does not replace configuration management like Chef or Puppet. In fact, when building images, Packer is able to use tools like Chef or Puppet to install software onto the image.

What is in this project

In this project, we will be baking an AMI using Packer and doing configuration using ansible during the baking process. A static website will be deployed. And the same ansible code is being used to provision the static website using nginx during this whole baking process. Tags will be assigned to AMI during baking process. It is intended to identify the latest AMI available and have it ready for EC2 instance creation. Subnet ID is supposed to be collected for packer builder so that it is able to create AMI using the Subnet ID. Here we are going to manage our terraform code in two different sections: one is for creating VPC, subnet, and other network information, the other is for grouping EC2 instance inside our network using AMI created by the packer previously.

Notes: We need to start and enable nginx using systemctl start and enable to ensure that inginx will be ready to process spontaneously.

Installing Terraform

For macOS, install Terraform using Homebrew. Check the version to confirm if it is installed correctly.

$ brew install terraform
$ terraform --version

Note: Brew needs to be preinstalled

To install Linuxbrew on your Linux distribution, fist you need to install following dependencies as shown.

--------- On Debian/Ubuntu --------- 
$ sudo apt-get install build-essential curl file git
--------- On Fedora 22+ ---------
$ sudo dnf groupinstall 'Development Tools' && sudo dnf install curl file git
--------- On CentOS/RHEL ---------
$ sudo yum groupinstall 'Development Tools' && sudo yum install curl file git

Once the dependencies installed, you can use the following script to install Linuxbrew package in /home/linuxbrew/.linuxbrew (or in your home directory at ~/.linuxbrew) as shown.

$ sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"

Next, you need to add the directories /home/linuxbrew/.linuxbrew/bin (or ~/.linuxbrew/bin) and /home/linuxbrew/.linuxbrew/sbin (or ~/.linuxbrew/sbin) to your PATH and to your bash shell initialization script ~/.bashrc as shown.

$ echo 'export PATH="/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin/:$PATH"' >>~/.bashrc
$ echo 'export MANPATH="/home/linuxbrew/.linuxbrew/share/man:$MANPATH"' >>~/.bashrc
$ echo 'export INFOPATH="/home/linuxbrew/.linuxbrew/share/info:$INFOPATH"' >>~/.bashrc

Then source the ~/.bashrc file for the recent changes to take effect.

$ source  ~/.bashrc

Check the version to confirm if it is installed correctly.

$ brew --version

For Windows, install Terraform using Chocolatey. Check the version to confirm if it is installed correctly.

$ choco install terraform
$ terraform --version

Manual installation for macOS, make a “terrform” directory under ~/Downloads find the appropriate package for your system and download it as a zip archive, unzip the package. Then move the file to /usr/local/bin/terraform Then vim ~/etc/profile and edit at the end of the file. Check the version to confirm if it is installed correctly.

$ mkdir terraform 
$ mv ~/Downloads/terraform /usr/local/bin/terraform
$ vim cd /etc/profile

Profile

export $PATH="$PATH:/usr/local/bin/terraform"

Check the version to confirm if it is installed correctly.

$ terraform --version

Manual installation for Windows, this stack overflow article contains instructions for setting the PATH on Windows through the user interface.

Check the version to confirm if it is installed correctly.

$ terraform --version

Installing Packer

At this point, we already have brew installed, so that we will take this installation in an easy manner. However, you are free to explore other installation methods here.

Homebrew

If you’re using OS X and Homebrew, you can install Packer by running:

$ brew install packer

Chocolatey

If you’re using Windows and Chocolatey, you can install Packer by running:

choco install packer

Notes: Homebrew uses Git for downloading updates and contributing to the project, which is very helpful to install multiple tools seamlessly. You are interested in learning more about Homebrew, please visit Brew.

Verify packer installation

$ packer --version
1.5.5

Installing Ansible

$ sudo yum install ansible

Verify Ansible

$ ansible --version
ansible 2.9.7
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/cloud_user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Aug  7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]

Creating user and configure AWS

Based on AWS best practice, root user should not be used to perform any task. So that we need to login to root user and create a user with suitable policies.

Login as a Root user

Create a user under IAM service

Choose programmatic access

Attach required policies

Create user without tags

Keep credentials (Access key ID and Secret access key)

Now we will get our hands dirty!

Step 1: Setup a network using Terraform

In this step, we are going to create a VPC with public subnet along with key pair which we use to ssh in all EC2 instances. We output subnet id which we need to place in packer builder using which packer can create an AMI used in this VPC.

How it works

vim ~/ImmutableInfrastructure/networkTerraform/resources.tf

How it works

vim ~/ImmutableInfrastructure/networkTerraform/resources.tf

terraform init, terraform validate (errors may occur, please fix errors provided by terraform) and terraform apply (provide with both access_key and secret_key)

$ terraform init

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.59.0...

The following providers do not have any version constraints in configuration,
so the latest version was installed.
$ terraform validate
$ terraform apply
var.access_key
  Enter a value: XXXXXXXXX // Please provide your access_key

var.secret_key
  Enter a value: XXXXXXXXX // Please provide your secret_key

Keypair, vpc and subnet created after terraforming resources.tf in networkTerraform folder

How it works

cd ~/ImmutableInfrastructure/terraform/modules/instance

terraform init, terraform validate (errors may occur, please fix errors provided by terraform) and terraform apply (provide with both access_key and secret_key)

$ terraform init

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.59.0...

The following providers do not have any version constraints in configuration,
so the latest version was installed.
$ terraform validate
$ terraform apply
var.access_key
  Enter a value: XXXXXXXXX // Please provide your access_key

var.secret_key
  Enter a value: XXXXXXXXX // Please provide your secret_key

Ec2 with eip created after terraforming resources.tf in instance folder

Step 2: Create AMI using packer and ansible inside the above-created network

We are going to use our ansible configuration which installs nginx and setup static page. We enable nginx using systemctl so when an EC2 instance is created using this AMI, we have nginx started and ready to process incoming HTTP requests.

How it works

Packer build

cd ~/ImmutableInfrastructure/packer

$ packer build packer.json
.
. // The building process may take few minutes, please be patient!
.
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-2: ami-0ea25761c768c04b5

[cloud_user@lightninglife2c packer]$

Double confirm AMI in AWS console

Ami verification in AWS console

Step 3: Setup EC2 instance inside the network with packer AMI

We are going to read network state file using data resource so that we can use network created resources during instance creation.

data "terraform_remote_state" "network" {
  backend = "local"  config {
    path = "../networkTerraform/terraform.tfstate"
  }
}

Next, we need to find the latest created AMI which is available with our created tag so that it can be used during instance creation.

data "aws_ami" "ec2-ami" {
  filter {
    name   = "state"
    values = ["available"]
  }  filter {
    name   = "tag:Name"
    values = ["Packer-Ansible"]
  }  most_recent = true
}

Once we have all this information, we can simply use this information during EC2 instance creation.

How it works

cd ~/ImmutableInfrastructure/terraform

$ terraform init

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.59.0...

The following providers do not have any version constraints in configuration,
so the latest version was installed.
$ terraform validate
$ terraform apply
var.access_key
  Enter a value: XXXXXXXXX // Please provide your access_key

var.secret_key
  Enter a value: XXXXXXXXX // Please provide your secret_key

Once this code is executed we output elastic IP address assign to our EC2 instance. We use this IP address in our browser to confirm that our static website is working fine.

Voila, C’est fini!

Success!

My takeaway from this project:

This project covers almost every aspect of using terraform to create VPC and related resources.

In terms of terraform, it provides us with Iac (Infrastructure as Code), which lays a solid foundation for documentation and editability in the future. As we dicussed at the very beginning, to rebuild the infrstructure from scratch is not cost-effective and very much time-consuming. With Terraform, every file and element could be brought to table and in use in a few minutes. Apart from this, terraform validate also allows us to identify issues prior to VPC building process. Errors associated with detailed description gives us a good sense of probmes we may encounter, which is serving as an effective tool to debug errors and contributing to the ultimate solutions.

In terms of packer, it packs both terraform and ansible together. With a simple line of $ packer build packer.json, we are able to extract elements from terrafom and ansible so that we build up AMI with the click of a button. More importantly, it is compatible with varity of tools.

In terms of ansible, this project showcases how effective it can be in order to install nginx and build up a static html website page. With packer accompanied, ansible is shining with its feature of automation.

In terms of AWS, it truly backs up the acknowledge why it is the leading power in cloud industry. By using AWS CLI, we are able to build up VPC infrastructure seamlessly. Besides, by following the best practice of AWS, security is being put the priority. For instance, we do not login as root on a daily basis. Instead, users attached with appropriate policies are in place to guarantee no user has access to more resources that he or she may require. With that said, the intentional or unintentional deletion could be averted. Moreover, by using CLI, we significantly reduce the amount of time to build up our infrastructure. Rather than clicking tens of buttons, we only type in few command lines. So that our project can’t be fulfilled in a timely and effective manner.

At the end of the project, I will provide the link for all files required for this project for your convenience. Please visit https://github.com/lightninglife/ImmutableInfrastructure