Advanced AWS Infrastructure Deployment and Management with Terraform: A Comprehensive Guide

Introduction

Amazon Web Services (AWS) offers a vast landscape of cloud services that can cater to various needs, from simple storage solutions to complex networking configurations. Terraform, a powerful Infrastructure as Code (IaC) tool, enables you to define and provision a cloud infrastructure using a simple, declarative language. This article will guide you through an intermediate-level project integrating multiple AWS services using Terraform.

Understanding the Project Scope

Our project will involve setting up a robust web application environment. We’ll use the following AWS services:

1. Amazon EC2: For hosting our web servers.
2. Amazon RDS: To manage our database needs.
3. Amazon S3: For object storage.
4. AWS VPC: To configure a custom virtual network.
5. AWS IAM: For managing access and permissions.

Prerequisites

• AWS account and CLI set up with appropriate permissions.
• Terraform installed on your local machine.

Project Structure

Our Terraform project will be organized into modules for better maintainability and readability:

aws-terraform-advanced/
├── main.tf
├── variables.tf
├── outputs.tf
└── modules/
    ├── ec2/
         └── main.tf
         └── variables.tf
         └── outputs.tf
    ├── rds/
         └── main.tf
         └── variables.tf
         └── outputs.tf
    ├── s3/
         └── main.tf
         └── variables.tf
         └── outputs.tf
    ├── vpc/
         └── main.tf
         └── variables.tf
         └── outputs.tf
    └── iam/
         └── main.tf
         └── variables.tf
         └── outputs.tf

Step 1: Setting Up the AWS VPC Module

A VPC (Virtual Private Cloud) enables you to launch AWS resources in a virtual network.

modules/vpc/main.tf

resource "aws_vpc" "main" {
  cidr_block = var.cidr_block
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags = {
    Name = "main-vpc"
  }
}

resource "aws_subnet" "subnet" {
  vpc_id     = aws_vpc.main.id
  cidr_block = var.subnet_cidr
  availability_zone = var.availability_zone

  tags = {
    Name = "main-subnet"
  }
}

resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "main-gateway"
  }
}

resource "aws_route_table" "rt" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.gw.id
  }

  tags = {
    Name = "main-route-table"
  }
}

resource "aws_route_table_association" "a" {
  subnet_id      = aws_subnet.subnet.id
  route_table_id = aws_route_table.rt.id
}

Let’s go through each resource and understand its purpose and the significance of the defined values.

AWS VPC Resource (aws_vpc)

resource "aws_vpc" "main" {
  cidr_block = var.cidr_block
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags = {
    Name = "main-vpc"
  }
}

• Purpose: This resource creates a new Virtual Private Cloud (VPC) in AWS. A VPC is a virtual network dedicated to your AWS account, isolated from other virtual networks in the AWS cloud. It allows you to launch AWS resources into a network that you define.
• Values:
• cidr_block: Specifies the IP address range for the VPC, defined by a variable var.cidr_block. CIDR blocks define the range of IP addresses available in the VPC.
• enable_dns_support and enable_dns_hostnames: These options enable DNS resolution and DNS hostnames within the VPC, allowing you to use domain names to communicate with instances rather than IP addresses.
• tags: Assigns a name to the VPC for easier identification.

AWS Subnet Resource (aws_subnet)

resource "aws_subnet" "subnet" {
  vpc_id     = aws_vpc.main.id
  cidr_block = var.subnet_cidr
  availability_zone = var.availability_zone
  tags = {
    Name = "main-subnet"
  }
}

• Purpose: This resource creates a subnet within the VPC. Subnets enable you to partition the VPC’s IP address range into smaller segments and allocate them within different parts of the AWS network.
• Values:
• vpc_id: Associates the subnet with a VPC, in this case, the VPC created above.
• cidr_block: Defines the subnet's IP address range.
• availability_zone: Specifies the Availability Zone in which the subnet resides.
• tags: Names the subnet for identification.

AWS Internet Gateway Resource (aws_internet_gateway)

resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.main.id
  tags = {
    Name = "main-gateway"
  }
}

• Purpose: This resource creates an Internet Gateway and attaches it to the VPC. An Internet Gateway allows communication between the VPC and the internet, enabling instances in the VPC to access or be accessed from the internet.
• Values:
• vpc_id: Links the Internet Gateway to the specified VPC.
• tags: Provides a name for the Internet Gateway.

AWS Route Table Resource (aws_route_table)

resource "aws_route_table" "rt" {
  vpc_id = aws_vpc.main.id
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.gw.id
  }
  tags = {
    Name = "main-route-table"
  }
}

• Purpose: This resource creates a routing table for the VPC. Routing tables define rules, known as routes, which determine where network traffic from your subnet or gateways is directed.
• Values:
• vpc_id: Associates the route table with the VPC.
• route: Defines a route in the table; here, it directs all traffic (0.0.0.0/0) to the Internet Gateway.
• tags: Names the route table.

AWS Route Table Association (aws_route_table_association)

resource "aws_route_table_association" "a" {
  subnet_id      = aws_subnet.subnet.id
  route_table_id = aws_route_table.rt.id
}

• Purpose: This resource associates a subnet with a specific route table, which in turn determines the routing of network traffic for that subnet.
• Values:
• subnet_id: Specifies the subnet to be associated with the route table.
• route_table_id: Identifies which route table the subnet should be associated with.

modules/vpc/variables.tf

variable "cidr_block" {
  description = "The CIDR block for the VPC"
  default     = "10.0.0.0/16"
}

variable "subnet_cidr" {
  description = "The CIDR block for the subnet"
}

variable "availability_zone" {
  description = "The availability zone to use"
}

Step 2: Creating the EC2 Module

This module will define our EC2 instances for the web server.

modules/ec2/main.tf

resource "aws_security_group" "sg" {
  vpc_id = var.vpc_id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = [var.ssh_access_cidr]
  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "web-server-sg"
  }
}

resource "aws_instance" "web" {
  ami           = var.ami_id
  instance_type = var.instance_type
  subnet_id     = var.subnet_id
  security_groups = [aws_security_group.sg.name]
}

Let’s go through each resource and understand its purpose and the significance of the defined values.

AWS Security Group Resource (aws_security_group)

resource "aws_security_group" "sg" {
  vpc_id = var.vpc_id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = [var.ssh_access_cidr]
  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "web-server-sg"
  }
}

• Purpose: A security group acts as a virtual firewall for EC2 instances to control incoming and outgoing traffic. In AWS, security groups are associated with instances and provide security at the protocol and port access level.
• Values:
• vpc_id: Associates the security group with a specific VPC.
• egress: Defines rules for outgoing traffic. Here, it allows all outbound traffic from the instance.
• ingress: Defines rules for incoming traffic. Two rules are defined:
• SSH access (port 22) from a specified CIDR block (var.ssh_access_cidr).
• HTTP access (port 80) from any IP address (0.0.0.0/0).
• tags: Provides a name for the security group for easier identification.

AWS EC2 Instance Resource (aws_instance)

resource "aws_instance" "web" {
  ami           = var.ami_id
  instance_type = var.instance_type
  subnet_id     = var.subnet_id
  security_groups = [aws_security_group.sg.name]
}

• Purpose: This resource creates an EC2 instance, which is a virtual server in AWS’s Elastic Compute Cloud (EC2) for running applications on the AWS infrastructure.
• Values:
• ami: Specifies the Amazon Machine Image (AMI) ID for the instance, determining the OS and software configuration.
• instance_type: Defines the type of instance (size, capacity, etc.).
• subnet_id: Specifies the subnet within which the instance is launched.
• security_groups: Associates the instance with the defined security group to control traffic to and from the instance.

The aws_security_group resource is creating a security group with specific rules for network traffic, and the aws_instance resource is creating an EC2 instance with this security group attached. The security group's rules are essential for determining what kind of traffic is allowed to and from the instance, ensuring that the instance's network access is secure and controlled according to the specified requirements.

modules/ec2/variables.tf

variable "ami_id" {
  description = "The AMI id to use for the EC2 instance"
}

variable "instance_type" {
  description = "The instance type of the EC2 instance"
}

variable "subnet_id" {
  description = "The ID of the subnet"
}

variable "ssh_access_cidr" {
  description = "CIDR block for SSH access"
}

variable "vpc_id" {
  description = "The VPC ID where the instance is launched"
}

Step 3: Setting Up the RDS Module

RDS (Relational Database Service) will manage our database.

modules/rds/main.tf

resource "aws_db_instance" "db" {
  allocated_storage    = var.allocated_storage
  storage_type         = var.storage_type
  engine               = var.engine
  engine_version       = var.engine_version
  instance_class       = var.instance_class
  name                 = var.db_name
  username             = var.db_username
  password             = var.db_password
  parameter_group_name = var.parameter_group_name
  db_subnet_group_name = aws_db_subnet_group.db_subnet_group.name

  vpc_security_group_ids = [var.db_security_group_id]

  tags = {
    Name = "MyDBInstance"
  }

  backup_retention_period = var.backup_retention_period
  skip_final_snapshot     = var.skip_final_snapshot
}

resource "aws_db_subnet_group" "db_subnet_group" {
  name       = "my-db-subnet-group"
  subnet_ids = var.subnet_ids

  tags = {
    Name = "My database subnet group"
  }
}

This Terraform code snippet defines two resources related to Amazon Relational Database Service (RDS): aws_db_instance for creating an RDS database instance and aws_db_subnet_group for defining a DB subnet group. Let's break down each resource:

Resource: aws_db_instance

resource "aws_db_instance" "db" {
  allocated_storage    = var.allocated_storage
  storage_type         = var.storage_type
  engine               = var.engine
  engine_version       = var.engine_version
  instance_class       = var.instance_class
  name                 = var.db_name
  username             = var.db_username
  password             = var.db_password
  parameter_group_name = var.parameter_group_name
  db_subnet_group_name = aws_db_subnet_group.db_subnet_group.name
  vpc_security_group_ids = [var.db_security_group_id]
  tags = {
    Name = "MyDBInstance"
  }
  backup_retention_period = var.backup_retention_period
  skip_final_snapshot     = var.skip_final_snapshot
}

• Purpose: This resource creates and manages an RDS database instance.
• Values:
• allocated_storage: The size of the database storage (in gigabytes).
• storage_type: The type of storage (e.g., standard, gp2, io1).
• engine: The database engine (e.g., MySQL, PostgreSQL).
• engine_version: The version of the database engine.
• instance_class: The compute and memory capacity of the instance.
• name: The DB name (used by the engine to create a database).
• username & password: Credentials for accessing the database.
• parameter_group_name: Name of the DB parameter group to associate.
• db_subnet_group_name: A subnet group to associate with the DB instance, defined for better control over subnets.
• vpc_security_group_ids: A list of VPC security group IDs to associate.
• tags: Key-value pairs for resource identification.
• backup_retention_period: Number of days to retain backups.
• skip_final_snapshot: Determines whether to skip the creation of a final snapshot when the DB instance is deleted.

Resource: aws_db_subnet_group

resource "aws_db_subnet_group" "db_subnet_group" {
  name       = "my-db-subnet-group"
  subnet_ids = var.subnet_ids
  tags = {
    Name = "My database subnet group"
  }
}

• Purpose: This resource creates a DB subnet group which defines which subnets a database instance is allowed to be in.
• Values:
• name: The name of the DB subnet group.
• subnet_ids: A list of VPC subnet IDs for the subnet group.
• tags: Key-value pairs for resource identification.

The aws_db_instance resource provisions and manages the settings of an RDS database instance in AWS, while the aws_db_subnet_group resource is used to define a group of subnets in which RDS can operate. These configurations are essential for creating a managed relational database in AWS, ensuring that it is properly sized, configured, and located within the appropriate network.

modules/rds/variables.tf

We define variables needed for the RDS configuration, providing flexibility and customization for the database setup.

variable "allocated_storage" {
  description = "The allocated storage in gigabytes"
  type        = number
}

variable "storage_type" {
  description = "The type of storage (gp2, io1, standard)"
  type        = string
}

variable "engine" {
  description = "The database engine to use (e.g., mysql, postgresql)"
  type        = string
}

variable "engine_version" {
  description = "The engine version to use"
  type        = string
}

variable "instance_class" {
  description = "The instance type of the RDS instance"
  type        = string
}

variable "db_name" {
  description = "The name of the database to create when the DB instance is created"
  type        = string
}

variable "db_username" {
  description = "Username for the database"
  type        = string
}

variable "db_password" {
  description = "Password for the database"
  type        = string
}

variable "parameter_group_name" {
  description = "The name of the parameter group to associate with this DB instance"
  type        = string
}

variable "db_security_group_id" {
  description = "The VPC Security Group ID for the RDS instance"
  type        = string
}

variable "backup_retention_period" {
  description = "The days to retain backups for"
  default     = 7
  type        = number
}

variable "skip_final_snapshot" {
  description = "Determines whether a final DB snapshot is created before the DB instance is deleted"
  default     = true
  type        = bool
}

variable "subnet_ids" {
  description = "A list of VPC subnet IDs"
  type        = list(string)
}

Step 4: Implementing the S3 Module

S3 buckets will store static assets or files.

modules/s3/main.tf

resource "aws_s3_bucket" "storage" {
  bucket = var.bucket_name
  acl    = "private"

  lifecycle_rule {
    id      = "log"
    enabled = true

    transition {
      days          = 30
      storage_class = "STANDARD_IA" # Infrequent Access
    }

    expiration {
      days = 90
    }
  }

  tags = {
    Name = "MyTerraformBucket"
  }
}

• Purpose: This resource is used to create and manage an S3 bucket on Amazon Web Services. S3 buckets are storage containers in Amazon’s Simple Storage Service (S3) that can hold an unlimited amount of data.
• Values:
• bucket: Specifies the name of the S3 bucket. This name is globally unique across all AWS accounts.
• acl: Sets the Access Control List (ACL) for the bucket. The value "private" means that access to the bucket and its contents are restricted to the AWS account that created the bucket.
• lifecycle_rule: Defines rules for automating the management of objects within the bucket:
• id: A unique identifier for the lifecycle rule.
• enabled: Indicates whether the lifecycle rule is active.
• transition: Specifies when objects are transitioned to another storage class. In this case, objects are moved to the STANDARD_IA (Infrequent Access) storage class after 30 days, which is generally cheaper but with a retrieval fee.
• expiration: Specifies when objects should be automatically deleted. Here, objects are set to expire 90 days after creation.
• tags: Key-value pairs used for resource identification and categorization.

The aws_s3_bucket resource in this Terraform code is used to create an S3 bucket with a specific set of configurations. It includes an ACL setting for privacy, lifecycle rules for managing objects in the bucket efficiently, and tags for identification. The lifecycle rules in particular help manage costs and data retention policies by automatically transitioning and expiring objects based on defined schedules.

Step 5: Configuring the IAM Module

IAM roles and policies will manage permissions.

modules/iam/main.tf

resource "aws_iam_role" "example_role" {
  name = "example_role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_policy" "example_policy" {
  name        = "example_policy"
  description = "A test policy"
  policy      = var.policy_json
}

resource "aws_iam_role_policy_attachment" "test-attach" {
  role       = aws_iam_role.example_role.name
  policy_arn = aws_iam_policy.example_policy.arn
}

The provided Terraform code snippet defines three AWS IAM (Identity and Access Management) resources: aws_iam_role, aws_iam_policy, and aws_iam_role_policy_attachment. Each of these resources serves a specific purpose in managing access and permissions within AWS. Let's break down each resource:

Resource: aws_iam_role

resource "aws_iam_role" "example_role" {
  name = "example_role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

• Purpose: This resource creates an IAM role which can be assumed by specified AWS services or entities. IAM roles are used to delegate permissions without the need to share security credentials.
• Values:
• name: The name of the IAM role.
• assume_role_policy: This is a policy that defines who or what is allowed to assume the role. In this case, it's configured to allow an EC2 service to assume this role.
• Version: Policy language version.
• Statement: Policy statement(s) containing permissions.
• Action: Specifies the action that is allowed or denied (here, sts:AssumeRole).
• Effect: Determines if the action is allowed (Allow) or denied (Deny).
• Principal: Specifies the AWS service (in this case, EC2) that is allowed to assume the role.

Resource: aws_iam_policy

resource "aws_iam_policy" "example_policy" {
  name        = "example_policy"
  description = "A test policy"
  policy      = var.policy_json
}

• Purpose: This resource creates a standalone IAM policy, which is a document that explicitly lists permissions.
• Values:
• name: The name of the policy.
• description: A brief description of what the policy does.
• policy: The policy document, defined here as a variable (var.policy_json), which should contain the actual JSON policy text.

Resource: aws_iam_role_policy_attachment

resource "aws_iam_role_policy_attachment" "test-attach" {
  role       = aws_iam_role.example_role.name
  policy_arn = aws_iam_policy.example_policy.arn
}

• Purpose: This resource attaches the specified IAM policy to the specified IAM role.
• Values:
• role: The name of the IAM role to attach the policy to.
• policy_arn: The Amazon Resource Name (ARN) of the IAM policy to be attached.

These resources collectively define an IAM role with a specific trust policy (allowing EC2 to assume the role), create an IAM policy with defined permissions, and then attach this policy to the previously created role. This setup is a common pattern in AWS to grant specific permissions to AWS services or resources, allowing for secure and fine-grained access control.

modules/iam/variables.tf

Add a variable for policy JSON.

variable "policy_json" {
  description = "The JSON policy to attach"
}

Step 6: Integrating Modules in the Main Configuration

Now, we integrate these modules into our main Terraform configuration.

main.tf

provider "aws" {
  region = "us-west-2"  # AWS region
}

# VPC Module
module "vpc" {
  source             = "./modules/vpc"
  cidr_block         = "10.0.0.0/16"
  subnet_cidr        = "10.0.1.0/24"
  availability_zone  = "us-west-2a"
}

# EC2 Module
module "ec2" {
  source             = "./modules/ec2"
  ami_id             = "ami-0c55b159cbfafe1f0"  # Example AMI ID
  instance_type      = "t2.micro"
  subnet_id          = module.vpc.subnet_id
  vpc_id             = module.vpc.vpc_id
  ssh_access_cidr    = "0.0.0.0/0"
}

# RDS Module
module "rds" {
  source               = "./modules/rds"
  allocated_storage    = 20
  storage_type         = "gp2"
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t2.micro"
  db_name              = "mydatabase"
  db_username          = "dbuser"
  db_password          = "securepassword"
  parameter_group_name = "default.mysql5.7"
  db_security_group_id = module.vpc.security_group_id
  backup_retention_period = 7
  skip_final_snapshot     = true
  subnet_ids            = module.vpc.subnet_ids
}

# S3 Module
module "s3" {
  source         = "./modules/s3"
  bucket_name    = "my-terraform-bucket"
}

# IAM Module
module "iam" {
  source          = "./modules/iam"
  policy_json     = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"s3:*\",\"Resource\": \"*\"}]}"
}

outputs.tf

output "vpc_id" {
  value = module.vpc.vpc_id
}

output "ec2_instance_id" {
  value = module.ec2.instance_id
}

output "rds_instance_address" {
  value = module.rds.db_instance_address
}

output "s3_bucket_arn" {
  value = module.s3.bucket_arn
}

output "iam_role_arn" {
  value = module.iam.role_arn
}

Conclusion

This project showcases how to manage a multi-faceted AWS environment using Terraform. With this setup, you can deploy a robust web application with a dedicated network, compute resources, database, storage, and secure access management. Remember, this is a starting point; real-world applications may require additional configurations for security, scalability, and compliance.