Terraforming Resources into Modules

I dove this week into Modules, an important layer of abstraction for turning Infrastructure into Code.

Last week, I learned Resources, which are the building blocks of Terraform. A Resource is a single infrastructure component, such as an EC2 instance. It has properties that define its desired state, such as the AMI ID. Terraform uses this information to create and manage the specified Resource.

A Terraform Module, on the other hand, is a collection of Terraform files that is managed holistically. A Module can accept input variables and return output values, allowing it to be used in different parts of your Terraform code and across multiple configurations. Modules allow you to organize and reuse your Terraform code, making it more modular and maintainable.

Module Project

To practice building out these more modular abstractions, I took a Resource and transformed it into three Terraform Modules. The starting Resource can be found on GitHub here:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }

  required_version = ">= 0.14.9"
}

provider "aws" {
  profile = "default"
  region  = "us-west-2"
}


resource "aws_instance" "app_server" {
  ami           = "ami-830c94e3"
  instance_type = "t2.micro"

  tags = {
    Name = "ExampleAppServerInstance"
  }
}

This Resource defines and configures an AWS EC2 instance.

The first block of code, starting with terraform, specifies the required providers and versions for this project. In this case, it uses a Hashicorp provider to interact with Amazon Web Services.

The aws_instance resource block defines an AWS EC2 instance. This resource has several properties, such as the AMI ID, instance type, and tags. It sets the AWS profile to "default" and the region to "us-west-2".

The aws_instance resource block defines an AWS EC2 instance. This resource has several properties, such as the AMI ID, instance type, and tags.

This is a nice building block to start with, but we don’t want just a part; we want a whole machine.

For my Project, I extracted the Resource code and turned it into a separate module file. I also added a bootstrap script to the template to transform the EC2 instances into NGINX servers. Furthermore, I created a Security Group module allowing HTTP, HTTPS, and SSH traffic in and out of instances and applied it to the EC2 instances. Lastly, I created and assigned an IAM role to the EC2 instance that allows it to read external data sources, such as the names of S3 buckets.

My file structure ended up looking like this:

Build the Root Module

This Terraform configuration uses a number of modules to create an AWS EC2 instance, an AWS security group, and an AWS IAM role and policy.

#Copy the provider from original ec2.tf file
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }

  required_version = ">= 0.14.9"
}

#Changed the region to us-east-1
provider "aws" {
  profile = "default"
  region  = "us-east-1"
}

#Replace the resource with a module
module "ec2module" {
  #References file location of new module
  source = "./ec2module"

  # Input variables for the module that will be used by resource in the module\
  # Updated the AMI id to one that still works
  ami = "ami-0b0dcb5067f052a63"
  instance_type = "t2.micro"
  instance_name = "example-app-server-instance"
  security_group_name = module.webaccessSG.security_group_name
  iam_instance_profile = module.listalls3bucketsIAM.instance_profile_name
}

# Specify the name for the security group
module "webaccessSG" {
  source = "./webaccessSG"
  security_group_name = "security-group"
}

#Creates IAM Policy, IAM Role, and Instance Profile to list all S3 buckets
module "listalls3bucketsIAM" {
  source = "./listalls3bucketsIAM"
  instance_profile_name = "my-instance-profile"
  role_name = "my-ec2-role"
}

The configuration specifies the provider to be used (AWS) and the required version of Terraform. It then uses the module block to specify three different modules: ec2module, webaccessSG, and listalls3bucketsIAM. The ec2module module is used to create an EC2 instance and the EC2 instance is associated with the security group and IAM role and policy. It references the webaccessSG and listalls3bucketsIAM modules to specify the security group and IAM role and policy that should be associated with the EC2 instance.

Included in the same directory it the outputs.tf:

#Return Instance ID and Public IP of the instance
output "instance_id" {
  value = module.ec2module.instance_id
}

output "public_ip" {
  value = module.ec2module.public_ip
}

This output block to returns the instance_id and public_ip of the EC2 instance created by the ec2module module. When you apply the configuration, the instance_id and public_ip of the EC2 instance will be displayed as output. This allows you to easily access the instance ID and public IP of the EC2 instance for further use in your Terraform configuration or elsewhere.

The ec2 Module

This module will create a private key, public key, and EC2 instance when you apply the configuration:

#Create a private Key that can be used in my_key_pair
resource "tls_private_key" "pk" {
  algorithm = "RSA"
  rsa_bits  = 4096
}

#Create a key pair
resource "aws_key_pair" "my_key_pair" {
  key_name   = "my_key_pair"
  
  # Use the tls_private_key to create a public key on AWS
  public_key = tls_private_key.pk.public_key_openssh

  # Create a "myKey.pem" to your computer
  provisioner "local-exec" { 
    command = "echo '${tls_private_key.pk.private_key_pem}' > ./my_key_pair.pem"
  }
}

resource "aws_instance" "app_server" {
  #Import the instance_type, ami, and tag using variables declared in the main directory main.tf and the variables.tf in the ec2module directory
  ami           = var.ami
  instance_type = var.instance_type
  iam_instance_profile = var.iam_instance_profile
  key_name = aws_key_pair.my_key_pair.key_name
  security_groups = [var.security_group_name]
  user_data = file("${path.module}/bootstrap.sh")
  tags = {
    Name = var.instance_name
  }
}

This configuration creates a private key using the tls_private_key resource, which can then be used to create a public key on AWS using the aws_key_pair resource. This public key is associated with an AWS EC2 instance, which is created using the aws_instance resource. The EC2 instance is created using variables specified in the main.tf and variables.tf files in the ec2module directory, as well as the bootstrap.sh script.

The input variables were defined in the root template and are declared locally in variables.tf:

# Input variables to transform the inputs from the main directory's main.tf file
variable "ami" {
  type = string
  description = "The AMI ID for the EC2 instance"
}

variable "instance_type" {
  type = string
  description = "The instance type for the EC2 instance"
}

variable "instance_name" {
  type = string
  description = "The name for the EC2 instance"
}

variable "security_group_name" {
  type = string
  description = "The name for the EC2 Security Group"
}

variable "iam_instance_profile" {
  type = string
  description = "The name for the EC2's Instance Profile"
}

The bootstrap.sh is an NGINX installation script found in the same directory:

#!/bin/bash
sudo yum update -y
sudo amazon-linux-extras install nginx1 -y 
sudo systemctl enable nginx
sudo systemctl start nginx

This Terraform configuration defines two output variables instance_id and public_ip in outputs.tf:

# Output variables for the module
output "instance_id" {
  value = aws_instance.app_server.id
  description = "The ID of the EC2 instance"
}

output "public_ip" {
  value = aws_instance.app_server.public_ip
  description = "The public IP address of the EC2 instance"
}

These outputs are the ID and public IP address of the the aws_instance created in main.tf

The listalls3bucketsIAM module

This configuration creates an IAM Role, attaches an IAM policy to the Role, and creates an IAM instance profile for an EC2 instance:

#Policy that allows you to read all S3 buckets
resource "aws_iam_policy" "s3_list_all_buckets" {
  name        = "s3-list-all-buckets"
  description = "Allow users or groups to list all S3 buckets in the account"
  policy =file("${path.module}/s3listallbucketspolicy")
}

# Create an IAM role that will access policy
resource "aws_iam_role" "ec2_role" {
  name = var.role_name
  assume_role_policy = file("${path.module}/assumerolepolicy")
}

# Attach the IAM policy to the IAM role
resource "aws_iam_role_policy_attachment" "s3_full_access" {
  role       = aws_iam_role.ec2_role.name
  policy_arn = aws_iam_policy.s3_list_all_buckets.arn
}

# Create an IAM instance profile for the EC2 instance that will use Role
resource "aws_iam_instance_profile" "ec2_profile" {
  name  = var.instance_profile_name
  role = aws_iam_role.ec2_role.name
}

The IAM role allows users to list all S3 buckets in the account, and the IAM instance profile allows the EC2 instance to use the IAM role. The configuration also includes variables for the names of the IAM role and instance profile.

The policies being attached are assumerolepolicy and s3listallbucketspolicy.

assumerolepolicy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

s3listallbucketspolicy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListAllMyBuckets"],
      "Resource": ["arn:aws:s3:::*"]
    }
  ]
}

The inputs are declared in variables.tf:

variable "instance_profile_name" {
  type = string
  description = "The name for the Instance Profile"
}

variable "role_name" {
  type = string
  description = "The name for Role"
}

The “instance_profile_name” variable is used to specify the name of an IAM instance profile, while the “role_name” variable is used to specify the name of an IAM role.

The output of the instance_profile_name is passed back to the root module from outputs.tf:

#Output the Instance Profile Name
output "instance_profile_name" {
  value = aws_iam_instance_profile.ec2_profile.name
  description = "The name of the Instance Profile"
}

This instance_profile_name will be used by the ec2module

The webaccessSG Module

Use this module to create an AWS Security Group that will provide HTTP and HTTPS access to everyone, and will allow the machine it’s deployed from to access it via SSH:

#Using a Terraform data source to find my current IP address
data "http" "ip" {
  url = "https://ifconfig.me/ip"
}

#Creating a Security Group with HTTP, HTTPS, and SSH access
#SSH access is for my current IP address only
resource "aws_security_group" "web_access" {
  name        = var.security_group_name
  description = "Allow HTTP and HTTPS web access, and SSH access from your current_ip"
  
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "HTTP web access"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  
  ingress {
    description = "HTTPS web access"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  
  ingress {
    description = "SSH Access from your local machine"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["${data.http.ip.response_body}/32"]
  }
}

It defines a Terraform data source that uses the “http” provider to make a request to the URL “https://ifconfig.me/ip" to retrieve the user’s current IP address. It then creates an AWS security group called “web_access” with ingress rules that allow HTTP, HTTPS, and SSH traffic. The SSH ingress rule is restricted to only the user’s current IP address, which is retrieved using the data source defined earlier.

The name of the Security Group is passed in from the root module and declared into variables.tf:

# Input variables for the webaccessSG module
variable "security_group_name" {
  type = string
  description = "The name for the security group"
}

The outputs passed back to the parent module are security_group_id and security_group_name:

# Output variables for the module
output "security_group_id" {
  value = aws_security_group.web_access.id
  description = "The ID of the security group"
}

output "security_group_name" {
  value = aws_security_group.web_access.name
  description = "The name of the security group"
}

The security_group_id output variable returns the ID of the AWS security group created by the module, and the security_group_name output variable returns the name of that security group. These output variables are used to reference the security group used by the ec2Module.

Wrap Up

In summary, modules are an effective way of organizing and reusing Terraform code across many projects. The configuration of the Instance Profiles and Security Groups is done so that they can easily be inserted into configurations for any resource requiring permissions similar to EC2 instances in this project.