Read more: https://cloudwithdj.com/using-terraform-to-deploy-a-web-server-and-run-a-bootstrap-script/

Terraform isn’t just for big, end-to-end projects. As a Cloud or DevOps engineer, you’ll often work on smaller but critical tasks like creating and connecting a database server, setting up a security group, or running a bootstrap script on a web server.

For beginners, these tasks are great practice because they reflect what you’ll do in enterprise environments. I created a step-by-step tutorial to walk you through some of the daily tasks you might encounter.

This tutorial walks you through deploying a simple AWS setup using Terraform. You’ll create a database server, a web server with a fixed IP address, configure security groups, and run a bootstrap script on the web server.

Read more: https://cloudwithdj.com/using-terraform-to-deploy-a-web-server-and-run-a-bootstrap-script/

If you’re diving into AWS with Python, Boto3 is the go-to library. It’s the official SDK that makes it easy to interact with AWS resources directly from your Python scripts. In this guide, I’ll walk through how to use Boto3 to handle various tasks with AWS Identity and Access Management (IAM).

With IAM, you can:

• Create new users
• Manage permissions
• Set up policies
• And so much more.

Prerequisite

Before we dive into the code, make sure you’ve got these set up:

• Python 3: Install it if you don’t already have it.
• Boto3: You can install it with pip: pip install boto3.
• AWS Credentials: You’ll need these to access your AWS account. See below:

How to create a user

When you set up a new AWS account, it’s a best practice to create a dedicated IAM user instead of using your root account for day-to-day tasks.

• create a python file createuser.py with the code below in the file

• Run python createuser.py
• You should see this below in your terminal.

• Go to your AWS console, you should see the user has been created.

How to List All IAM Users

• create a python file with the following code

• Run it. Here is the result.

How to Update a User

Need to rename an IAM user? AWS lets you update a user’s details with Boto3’s update_user function. This is especially useful when renaming users to align with new naming conventions or correcting a mistake.

• create a python file with this code

• Here is the result in your terminal.

• You should see the change in your aws console.

How to Create an IAM Policy

An IAM policy defines the actions a user or group can perform and the resources those actions apply to. With Boto3, you can create custom policies tailored to your specific access control needs. Let’s walk through creating a policy that grants access to specific actions in DynamoDB.

• Go to the IAM console. Click on poloicies and see new policy.

How to List All IAM Policies

To efficiently manage permissions in your AWS account, it’s essential to know which IAM policies exist. Using Boto3, we can retrieve a list of IAM policies, filtering for those created within your account.

How to Attach a Policy to a User

Attaching IAM policies to individual users is essential for granting specific permissions. Below is a step-by-step guide using Boto3 to attach a policy to a user.

How to Create a Group

When managing permissions in AWS, creating groups helps streamline the process. Instead of attaching policies to individual users, you can attach policies to a group. All users added to the group inherit the permissions of the group’s policies.

How to Add a User to a Group

In AWS Identity and Access Management (IAM), users can be added to groups to inherit the permissions attached to the group. This is an efficient way to manage user permissions across your AWS account.

How to Attach an IAM Policy to a Group

Attaching a policy to a group allows all users in that group to inherit the permissions defined in the policy. This streamlines permission management for multiple users.

Find the python code here: https://github.com/djcloudking/aws-skills-challenges/blob/main/15_AWS%20IAM/attachpolicytogroup.py

How to Create an IAM Role

An IAM role is a secure way to grant permissions to AWS services or users. The creation of a role requires defining a trust relationship policy to specify which entities (such as AWS services) can assume the role.

Find the python code here: https://github.com/djcloudking/aws-skills-challenges/blob/main/15_AWS%20IAM/iamrole.py

How to Attach an IAM Policy to a Role

An IAM policy defines the permissions for an IAM role. You can attach policies to a role using the attach_role_policy method provided by the AWS SDK (boto3).

Find the python code here: https://github.com/djcloudking/aws-skills-challenges/blob/main/15_AWS%20IAM/attachiampolicytorole.py

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

Usually, you’d create a `main.tf` file and put all the resources needed to set up your EKS cluster in one place. But this time, you’ll be using Terraform modules instead.

Modules make things a lot easier to manage. For example, if you need to update inbound rules, you just go to `vpc.tf` instead of digging through `main.tf`. It keeps things organized and makes troubleshooting so much simpler. I actually wrote a blog about troubleshooting using terraform modules, check it out.

PREREQUESITES

• AWS account access
• Knowledge of Kubernetes

Step 1: Create the architecture

This structure shows the key components and their roles in deploying the EKS cluster:

EKS Cluster Deployment
├── VPC Module
│ ├── Subnets
│ ├── Route Tables
│ └── Internet Gateway
├── EKS Module
│ ├── Worker Nodes
│ └── Control Plane
├── Security Group Module
│ ├── Ingress Rules
│ └── Egress Rules
├── Variables
│ ├── VPC Configuration
│ ├── EKS Cluster Settings
│ └── Security Group Parameters
├── Versions
│ ├── Terraform Version
│ ├── AWS Provider Version
│ └── Kubernetes Provider Version
└── Outputs
├── VPC ID
├── EKS Cluster Name
└── Security Group IDs

Step 2: Install AWS CLI

• Follow the below link to Install AWS CLI.

https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Step 3: Install Terraform (on your local machine)

• Next, Install Terraform using the below link.

https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli

Step 4: Connect Terraform to AWS

• For Terraform to speak with AWS, you need AWS credentials. Download Access keys

Step 5: Create Terraform modules

These are the 6 files you will create to deploy your eks cluster:

• Create VPC module
• Create EKS module
• Create Security Groupe module
• Create variables
• Create version
• Create output

Find the modules in this repository: https://github.com/djcloudking/terraform-challenges/tree/main/11_Create%20a%20EKS%20Cluster%20using%20Terraform%20Modules

Step 6: Run Terraform

• Initialize Terraform
  Run terraform init. This step sets up the Terraform environment, downloading necessary modules, providers, and configurations.

• Review the Terraform Configuration (Optional)
  Use terraform plan to preview the infrastructure changes Terraform will apply. This helps ensure the configuration aligns with your expectations before deployment.
• Apply the Terraform Configuration
  Execute terraform apply to deploy the EKS cluster along with the VPC. Terraform will handle the creation process seamlessly.

Your EKS cluster has been deployed successfully. And the other resources stated above.

Step 7: Secure EKS cluster

• Go to EKS console. Click on your cluster.
• Go to access tab. Copy the access entry. If you don’t have one, click Create Access entry.
• Navigate back to the compute tab, you should see the node group and the desire state.
• You have successfully secure your EKS cluster, not everyone should get access to it.

That’s all, folks! Following the steps above will enable you to create EKS cluster using Terraform modules.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

Scenario

Consider your DevOps team using a development lab for application testing, where AWS EC2 instances in the lab run continuously. Your management raises concerns about the escalating costs, as these instances operate 24/7 even when the team isn’t working.

Solution

The task is clear: implement an automated solution that provides the necessary CI/CD environment while reducing unnecessary expenses.

To address this, we can automate instance management by using a Python script combined with AWS Lambda. The script will target EC2 instances tagged as “Dev,” stopping them outside of active work hours.

Prerequisites

• Python installed.
• AWS CLI installed
• AWS account with necessary IAM permissions.
• Knowledge of AWS Lambda, EC2, and Boto3 documentation

Step 1: Define the Scope

Since costs are primarily associated with development environments, we’ll focus on EC2 instances tagged “Dev” in our solution. This tag will enable our Python script to identify and manage only the relevant resources.

Below is a snapshot of the EC2 fleet used by our developers.

Step 2: Write the Python Script

Our Python script uses Boto3, the AWS SDK for Python, to interact with AWS services.

Script Flow:

• Imports Boto3 to interact with AWS.
• Creates an EC2 client for a specified region
• Fetches a list of all running instances.
• Filters for instances tagged “Dev” and stops them.
• Outputs the IDs of the stopped instances for verification.

Script Breakdown:

• Imports necessary modules.
• Identifies “Dev” instances using specific tag criteria.
• Stops the instances and confirms successful execution.

Before all of this, you should make sure your AWS is configured.

Step 3: Test the Script

• Run the script manually to verify it’s stopping only the desired “Dev” instances. You’ll see confirmation of which instances were stopped, displayed both in the console and the EC2 dashboard.

Verify from the console:

• Run the script again. Based on our script you should get an error message.

Step 4: Automate with AWS Lambda

It’s time to fully automate this process. Make sure we don’t need a human to run the python script.

First, create a Lambda function with your Python script.

• In the Lambda console, create a new function with Python as the runtime.

• Paste your Python script into the function’s code editor and save.

Second, configure AWS EventBridge to trigger this Lambda function on a schedule, such as weekdays at 7 p.m. after the team’s working hours.

• Use EventBridge to set a cron job to trigger the Lambda function according to your desired schedule.
• From our Lambda Function, copy the function ARN and Click “Add Trigger”

• Trigger has been created.

With Lambda and EventBridge, the script will run automatically, stopping the “Dev” instances on schedule, eliminating human error and further controlling costs.

Step 5: Test the Lambda Function

• Lastly, it’s time to test our Lambda function. To do so, let’s ensure that we have at least ONE Dev Instance running in our EC2 Instance Dashboard.

• Run the Lambda function to confirm it performs as expected. Check the EC2 dashboard to ensure that only “Dev” instances were stopped.

Challenge:

After testing lambda, I encountered this error message:

The error indicates that the Lambda function does not have permission to stop EC2 instances. The IAM role attached to the function does not have the necessary permissions to allow the ec2:StopInstances action on the specified instance.

My Solution:

If you don’t have permissions to do the following changes, contact your AWS Administrator. In my case, this what I did:

• Go to the IAM console and select the ‘Lambdaec2describerole’ role.
• Click on ‘Add permissions’ and then ‘Create inline policy’.

• In the JSON editor, add the following policy:

• Click ‘Review policy’ and give it a name like ‘AllowStopEC2Instance’.
• Click ‘Create policy’ to attach the new inline policy to the ‘Lambdaec2describerole’ role.
• Re-test your Lambda function, everything will work fine.

If you haven’t experienced this issue, it shouldn’t be a concern.

With this automated solution, you’ve successfully managed the cost of your CI/CD environment, ensuring it serves the team’s needs without unnecessary expense or oversight. This approach minimizes manual intervention, optimizes performance, and keeps your DevOps processes streamlined.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

This short tutorial is the perfect example of how to use CDK with Python to automate a workflow. Read below for more details.

Scenario

You have a photo-sharing application where users can upload images, and you’d like to automatically analyze each image as soon as it’s uploaded.

Once the image is uploaded to an S3 bucket, AWS Rekognition will be used to analyze the photo, identifying key details like objects and scenes. The results of this analysis should be stored in DynamoDB, allowing each user to view details about their uploaded photos. AWS Lambda will manage the process of analyzing the image and saving the results, and you’ll use AWS CDK in Python to set up this automated workflow.

Architecture

Project Outline

In this tutorial, we’re going to set up a simple workflow that automatically analyzes images whenever they’re uploaded to an S3 bucket. We’ll use AWS Rekognition to handle the image analysis, and the results will be saved in DynamoDB. AWS Lambda will take care of orchestrating everything behind the scenes, and we’ll use the AWS Cloud Development Kit (CDK) in Python to build it all.

Pre-requisite

• AWS account with admin rights
• Knowledge of CDK, Python, Javascript

Now, let’s have fun

Step 1. Install the CDK in our environment

• Go to you AWS console.
• Open Cloudshell.
• Run sudo npm install -g aws-cdk-lib

Cheat Sheet: directory name must be cdk-app/ to go with the rest of the tutorial, changing it will cause an error

• Create a directory mkdir cdk-app
• Move in the directory cd cdk-app/
• Now, initialize the application cdk init app-language

• Always verify if it works correctly cdk ls

Step 2. Copy the content of cdk-app-stack.js into lib/cdk-app-stack.js

• List the contents of lib/ and go inside the folder

• Remove the content: rm cdk-app-stack.js and create new content touch cdk-app-stack.js

• Open the new file: nano cdk-app-stack.js
• Copy the content of cdk-app-stack.js located in this code repository: https://github.com/djcloudking/python-challenges/blob/main/95_Python%20project%206/lib/cdk-app-stack.js and paste it in cloudshell
• Run cat cdk-app-stack.jsto verify if everything is there.

Step 3. Setup the Lambda function

• Back to cdk-app, verify if lambda function does not exist yet.

• If no, create a new lambda directory mkdir lambda && cd lambda
• Then, create a python file touch index.py. Edit it with nano index.py.
• Copy the python code in this repository: https://github.com/djcloudking/python-challenges/blob/main/95_Python%20project%206/lambda/index.py and paste it.

• Verify if the code has been copied using cat index.py

Step 4. Bootstrap the CDK application

• It’s time to bootstrap our cdk application cdk bootstrap
• Run cdk bootstrap . You should see a message asking you to specify an environment name.

• Go one level down with cd ..
• Run again cdk bootstrap

Step 5. Synthesize as a CloudFormation template

• Go to CloudFormation console and verify if something has been created

• Now, run cdk synth to target cloudformation template that is going to be created and deployed to the cloudformation stack.

Step 6. deploy the CDK stack

• Finally, deploy the cdk stack: cdk deploy
• If asked, enter Y to continue the deployment

• At the end of the deployment of the CDK, you will see a new stack in cloudformation

Step 7 — Verify S3 bucket and DynamoDB table

• Go to AWS S3 dashboard.
• Select a bucket and upload a picture in the bucket.

• If our CDK has been deployed correctly, we will see the description of this picture in our DynamoDB table.
• Go to the DynamoDB table. Click on explore items.

• As we intended, a Lambda function is triggered when an image is uploaded to the S3 bucket. Rekognition analyzes the picture and sends the results to DynamoDB.
• Let’s try again with different images.

By following this tutorial, you have successfully created a serverless solution that analyzes images using AWS Rekognition and stores the results in DynamoDB, all orchestrated via CDK in Python. This pipeline can be extended to handle more complex workflows or scaled for production use.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

Background

A three-tier architecture is a common framework for building scalable, reliable cloud applications. It breaks the application into three distinct layers, each serving a specific role:

• Presentation Layer (Web Tier): this layer is responsible for user interaction, managing the user interface (UI/UX), and processing HTTP/S requests and responses.
• Application Layer (App Tier): this layer handles the core business logic and processes data exchanges between the web and database layers.
• Data Layer (Database Tier): this layer stores, manages, and retrieves the application’s data, handling queries from the application layer.

This architecture promotes high availability, scalability, and clear separation of responsibilities, making it easier to maintain, scale, and optimize each layer independently.

Project Outline

Deploy an E-commerce Website on AWS in a three tier architecture using Terraform.

Prerequisite

• Access to AWS account.
• Install Terraform
• Familiarity with 3-tier-architecture

Let’s have fun.

Leveraging AWS services and Terraform allow you to efficiently build and maintain a robust, reliable platform that evolves alongside your business.

1. Create main.tf file and connect AWS to Terraform

• Go to your favorite IDE or open a terminal.
• Copy and paste the code below. Do not forget to change your profile name.

• Next, go to AWS S3 console. Create a S3 bucket.

I named my bucket terraform-ecom.

• Store Terraform state file in your S3 bucket by using the code below.

• It’s time to initialize our terraform code. Open a new terminal and run terraform init.

2- Create VPC

• Create two new files named variables.tf and vpc.tf respectively.
• In the variables.tf file, copy the code below to declare vpc variables.

• In the vpc.tf file, write your terraform code for vpc resource.

Cheat Sheet: After running your terraform code, verify your AWS if each resource is created correctly

3- Create Internet Gateway resource

• In the same vpc.tf, add the code lines for internet gateway.

4- Create Public subnet

• Back to your variables.tf file, create variables for public subnets

• In your vpc.tf file, add the two public subnets using terraform code.

4- Create Route Table

• In the same vpc.tf, create route table

• Associate public subnet to route table

5- Create Private subnet

• Still in vpc.tf file, complete your terraform code with private subnets

• It’s time to run terraform plan, then terraform apply.

• Go back to your AWS console. Verify if all the resources you mentioned in your vpc.tf file have been created.

6- Create Nat gateway

Leverage Terraform to deploy a NAT gateway, allowing outbound traffic from instances within private subnets.

• Create a natgateway.tf file.
• Copy and paste the following code lines in your file.

• After running terraform plan and terraform deploy, you should see Elastic IP address, and Nat gateways deployed in AWS.

7 — Create security groups

• Write a Terraform script to configure a security group, managing inbound and outbound traffic for instances.
• If you are having some issues to create security groups using terraform, find the code repository in this repo: https://github.com/djcloudking/terraform-challenges/blob/main/9_Deploy%20e-comerce%20website%20on%20AWS/security-group.tf
• Once again, verify if your security groups deployed correctly.

8- Create RDS

Use Terraform to set up a Relational Database Service (RDS) for efficient database management.

• First, create a rds.tf file.
• Next, create a database subnet group and a database instance.

• Go to your RDS dashboard and verify if it deployed.

• Finally, create a certificate manager.

9- Create Application Load Balancer

• Create alb.tf file

• create a target group

• Create a listener on port 80

• Create a listener on port 443

10- Create a SNS

• Create a sns.tf file.
• Copy and paste the code below in the file.

• Verify if SNS deployed in AWS.

11- Create Auto Scaling Group

Leverage Terraform to create an Auto Scaling Group.

• Find the code repository in this repo: https://github.com/djcloudking/terraform-challenges/blob/main/9_Deploy%20e-comerce%20website%20on%20AWS/asg.tf

12- Create Record Set in Route 53

• Create route53.tf file.
• Create a record set in Route 53

Following these steps will enable you to deploy a secure and scalable e-commerce website on AWS using Terraform. This guide offers a clear and structured process, ensuring that others can easily replicate your deployment.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

Background

Terraform is like a magic wand for managing your cloud infrastructure. Imagine writing a few lines of code, and voilà, your servers, databases, and networks are all set up automatically. That’s the power of Terraform.

Terrform module is a container for multiple resources that are used together. This can include configuration files, input variables, output values, and nested resources. Modules allow you to abstract your infrastructure into reusable components.

Prerequisite

For this project, you need:

• AWS account
• Terraform installed

Project Outline

When working with Terraform, you might encounter some challenges that can seem like they appear out of nowhere, growing by “leaps and bounds.” As a cloud engineer, this is one way to resolve these issues: the use of modules.

In our example, you notice that your team has built AWS infrastructure (EC2, VPC, S3, EBS, etc.) using Terraform. However, it seems that the EC2 instance is not being created when using the same Terraform template again. In the following tutorial, I will show you how to fix this issue and properly implement AWS EC2.

Let’s Have Fun

Instead of writing all the code in the main configuration file, I will use modules to separate the EC2 configuration, troubleshoot issues, and keep it ready for future implementation.

Verify if terraform and AWS are installed properly

• Open the terminal and type terraform --version

• Verify if AWS is configured properly. Type aws --version

Detach the EC2 configuration

• Detach the EC2 configuration from the main Terraform template.
• Create a new folder for your troubleshooting session.
• Inside the new folder, create a file named main.tf.
• Open main.tf and add the EC2 resource configuration.
• Add the provider definitions at the top of the main.tf file.
• Create variables to simplify the code.

If you leave the code as it is, you will encounter an error during execution. To fix this, create a new file named terraform.tfvars to define the values for ami and instance_type. Also,terraform.tfvars can be used to stored sensitive information.

Add some changes to our terraform code

• Update our Terraform code to use these variables. Creating a terraform.tfvars file ensures that any developers in the project can easily use and update it when needed to create new AWS EC2 resources.

Create terraform variables

• Create a new file named variables.tf to simplify and parameterize the code.
• Go back to main.tf, copy and cut all the variable definitions.

• Open variables.tf and paste the variable definitions into variables.tf.

Create the new EC2 resource

• Go to the terraform project and run terraform init

• Then terraform plan. Finally terraform apply. Type Yes.

Add more changes for a robust code

Now, let’s imagine that the code you are writing will be used by developers outside of your team. For that reason, I will add output and input.

• Run terraform destroy
• Create a new file named outputs.tf
• Add the following lines to outputs file:

• Run terraform init, terraform planand terraform apply

Create modules for EC2

To improve reusability and maintainability, we will create modules for EC2. This will be especially helpful when creating EC2 instances multiple times. By doing so, it resolves our issues.

• Run terraform destroy.
• Create a new folder called modules/ec2. This folder will contain all the necessary files for the EC2 module.
• Move the following files main.tf, variables.tf, terraform.tfvars and outputs.tf in the folder.
• After moving these files, delete terraform.tfvars. Other developers executing the module will create their own terraform.tfvars.

At this stage, any developer or team member can create a new main.tf file with the provider defined, from any location or repository, and execute the module to implement a new EC2 instance. Here’s how it will look:

• Finally, execute terraform plan and terraform apply to deploy the resources.
• Any developer can write a main.tf file, call the module they need, and add the necessary values to execute the code.

Voila! You’ve troubleshot the Terraform issue using a Terraform module. This approach also makes it easy to create reusable code for your Terraform automation.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

In this part, I will install Docker on AWS EC2 before deploying.

In part II, here what you will learn:

• Configuring AWS EC2 Userdata with Terraform
• Setting Up VSCode SSH Configuration
• Utilizing Terraform Provisioners
• Installing Docker on EC2 before deploying
• Replacing Resources and Deploying EC2 with Terraform
• Working with Terraform Variables
• Using Terraform Conditional Expressions
• Basics of Terraform Outputs

Part II:

Launching an AWS EC2 Instance with Terraform

• Google “AWS EC2 resource in terraform”. You should land on the section in the terraform documentation that mention AWS EC2.
• Go back to VS Code and add the block to main.tf in the Terraform directory.
• Enter the necessary information needed to create a VPC.

• Run terraform plan, then terraform apply -auto-approve. You should see a new resource.
• Reorganize the code in main.tf

Configuring AWS EC2 Userdata with Terraform

• Add userdata to our content to boostrap our instance. Boostrap goes over instance for docker to be ready.
• Create a new file named userdata.tpl

• Add it to the resource “aws_instance”

• Run terraform plan, then terraform apply -auto-approve. You should see a new resource.
• Go to EC2 dashboard to verify.

• Or run terraform state list

• Run terraform state show aws_instance.dev_node

• Copy and save the public IP address to connect “44.202.189.9”

• Connect to the instance:ssh -i ~/.ssh/keypairname username@ippublicaddress

• When connected, verify if docker is installed

Setting Up VSCode SSH Configuration

• Go to the extension, and search for “Remote — SSH”. Install it.

• Go back to explorer and type:cat ~/.ssh/config
• Create a file named “windows-ssh-config.tpl

• Create another file for linux named “linux-ssh-config.tpl

Utilizing Terraform Provisioners

• Go back to main.tf, under resource “aws_instance” enter this code

• Runterraform state list
• Runterraform apply-replace aws_instance.dev_node

• Enter Y.
• Go to your console and verify if itt’s working.

• Enter cat ~/.ssh/config, you should get a correct result.

Installing Docker on EC2 before deploying

• In VS code, go to view. Click on Command palette, then ssh. Check the IP address and click on it.

• Select Linux, and click OK.

• Open terminal and run docker
• Run docker --version

• Click “Yes, I trust the authors”.

• You’ve deployed a DEV environment using Terraform, and accesible via SSH remote. You can stop the lab here.

BONUS: Or you can continue the lab.

Let’s try to optimize what I did. I am going to modify couple of scripts to adjust everything.

Replacing Resources and Deploying EC2 with Terraform

• Modify the type of OS we are using. Change windows in the provisioner so that it can be useful when using other operating systems.

• Use the interpolation syntax: $ { } which is a calculated variable used
• Create a new file named “variables.tf”
• Go to the terraform document. Search for “Declaring an input variable”
• Copy the code and enter the necessary changes

• Save the file.
• Run terraform destroy -auto-approve. Enter windows when a value is requested.

Working with Terraform Variables

• The system is requesting to define var.host_os
• Create a new file named “terraform.tfars” with the following details

• Create a another file named “dev.tfars” with the following details

• Run the following command: terraform console -var-file=”dev.tfars”, then enter the variable

Using Terraform Conditional Expressions

• Run the following command: terraform console -var=”host_os=windows”, then enter the variable

• You realize you should set or change the interpreter by integrating variables.

• Run terraform plan, then runterraform apply -auto-approve.
• If nothing change, run terraform plan -refresh-only.

Basics of Terraform Outputs

• Launch terraform console, and type aws_instance.dev_node.public_ip to view IP address.
• Create a new file “outputs.tf”

• Run terraform apply -refresh-only.

• Run terraform output.to see the final result.

This is how you can optimize your terraform script with variables and outputs. This is the end.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

Scenario

My manager asked me to set up a new development environment. So, I decided to document the whole process of creating a secure and isolated AWS environment for our developers. I’ll also show you how to make it easily accessible using VS Code’s Remote SSH extension.

Architecture

Project Outline

Don’t forget! Our main goal is to create a secure and isolated AWS environment for developers, accessible through Remote SSH for streamlined workflow and enhanced security.

Here’s what you’ll learn:

Part I

• Setting up IAM role, keys and permission
• Setting Up Visual Studio Code for AWS and Terraform
• Initializing Terraform and Configuring the AWS Provider
• Applying Terraform to create an AWS VPC
• Utilizing Terraform State
• Applying Terraform to create AWS Subnets
• Creating an AWS Internet Gateway
• Applying Terraform to create AWS Route Tables
• Managing AWS Security Groups with Terraform
• Using Terraform Data Sources
• Applying Terraform to create an EC2 Key Pair
• Launching an AWS EC2 Instance with Terraform

Pre-requisite

• AWS account with the right access
• AWS CLI installed
• Terraform installed
• Visual studio Code used as IDE.

PART I

Setting up IAM role, keys and permission

• Go to IAM dashboard

• Click on users, then click create a user.

• Fill out the form with the necessary details. I chose “djdev” as my username.
• Do not check “provide user access to the AWS Management Console”. We need the “Access key — programmatic access”.
• In the Set permissions window, select attach policies directly.
• Search for and select Administrator Access in the permissions policies section. Leave tag as default.

• Now click on the user you created.

• Scroll down and click Security Credentials tab.
• Scroll down again and go to Access keys. Then click create access key.

• Select CLI for our programmatic access credentials.

• Check the bottom for confirmation. For tags, add what you want to describe what you’re doing.

• You’ve created your access key. Download the CSV file.

Setting Up Visual Studio Code for AWS and Terraform

• Open Visual Studio Code. Go to the extension. Search for AWS and install AWS Toolkit.

• Go to View, then Command Palette.
• Click on “AWS: create credential profile (djtest)”
• Copy the credentials from your CSV file and paste it in the credentials window.

• Save and close the window.
• Click Connect to AWS. You should see your profile.

Initializing Terraform and Configuring the AWS Provider

• Go back to the extension. Add Terraform and close the extension window.

• Create a new working directory or folder named Terraform. Open and select the folder in VS Code.

It’s now time to create our terraform files, starting with providers.tf, which is used by Terraform to interact with APIs for anything you want to build.

• Go into the Terraform folder and create a file named providers.tf
• Add the necessary code to the file

Cheat Sheet: I deleted the version so that it can download whatever version is newer.

• Look into the terraform documentation on how to add the provider block. Use the provider block with “shared credentials file”

• Copy and paste it under required provider
• Change shared credentials to “~/.aws/credentials” if you’re using Linux or “C:/Users/djkon/.aws/credentials” if you are using Windows and add your profile name.

• Open the terminal. Then run terraform init.

• Terraform has been successfully initialized.

Terraform Errors Example: Be careful about what you’re writing. You may encounter some errors, but read the error messages and try to resolve them

I resolved this issue by entering the right attribute and changing the profile name.

Applying Terraform to create an AWS VPC

• Google “AWS VPC resource in terraform”. You should land on the section in the terraform documentation that mention AWS VPC.
• Go back to VS Code and create a new file called main.tf in the Terraform directory.
• Enter the necessary information needed to create a VPC.

• Save the file and run terraform plan.

• Enter terraform apply, then yes.

• Next, click on AWS icon. Click No, and Resource. You should see your VPC.

• Go to the VPC Dashboard and verify if you see the VPC you created.

Utilizing Terraform State

Terraform use State to determine which changes to make to your infrastructure.

• The command terraform state list shows all the resources you created.

• The command terraform show displays everything in details.

• The command terraform destroy kills all existing resource.

Applying Terraform to create AWS Subnets

• Google again “AWS subnets resource in terraform”. You should land on the section in the terraform documentation that mention AWS subnets.
• Go back to VS Code and add the new code block to the main.tf in the Terraform directory.
• Enter the necessary information needed to create subnets.

• Ensure that our resource receive a public ip address. Add Map_public_ip_on_launch to label it correctly.
• Run terraform plan. You should see all the details that you applied.
• Run terraform apply -auto-approve. This new command will allow you to skip the yes or no question.

Terraform Errors Example: Be careful about what you’re writing. You may encounter some errors, but read the error messages and try to resolve them.

When I first executed this code:

I received this error message:

I fixed it by changing the AZ from us-east-1 to us-east-1a.

Creating an AWS Internet Gateway

• Repeat the same process between terraform documentation and main.tf
• Add the necessary information related to the infrastructure you want to create.

• Run terraform plan, then terraform apply -auto-approve. You should see a new resource, such as an internet gateway, in your resources and in your AWS account.

Applying Terraform to create AWS Route Tables

• Repeat the same process between terraform documentation and main.tf
• Add the necessary information related to the infrastructure you want to create.

• Run terraform plan, then terraform apply -auto-approve. You should see a new resource, such as an internet gateway, in your resources and in your AWS account.

• Now, let’s bridge the gap between route table and public subnet by providing a route table association.
• Apply the same process: search the resource in the terraform documentation, then copy the code block in the main.tf file
• Add the necessary information related to the infrastructure you want to create.

• Run terraform plan, then terraform apply -auto-approve.
• Verify if those resources were created.

Managing AWS Security Groups with Terraform

• Repeat the same process between terraform documentation and main.tf
• Add the necessary information related to the infrastructure you want to create.

• Run terraform plan, then terraform apply -auto-approve. You should see a new resource.

Using Terraform Data Sources

• Data source is a query from AMI to deploy a resource.
• Go to EC2 dashboard. Click on “Launch instance”.

• Click on Ubuntu AMI and copy AMI ID number.

• Go back to EC2 dashboard. Click on AMIs under Images. Select Public images. Then enter the same AMI ID in the search bar.

• Copy the AMI name and Owner.

• Repeat the same process between terraform documentation and vscode.
• Create a new file “datasources.tf”, enter this block code.
• Add the necessary information related to the infrastructure you want to create.

• Save the file.
• Run terraform plan, then terraform apply -auto-approve. You should see a new resource.

Applying Terraform to create an EC2 Key Pair

• Open the terminal and type ssh-keygen -t ed25519

• Name your keypair. I chose “dj_auth”. Do not add a passphrase.

• Enter sls ~/.ssh to see the directory.
• Repeat the same process between terraform documentation and main.tf, and use a terraform file function.
• Add the necessary information related to the infrastructure you want to create.

• Run terraform plan, then terraform apply -auto-approve. You should see a new resource.

Launching an AWS EC2 Instance with Terraform

You’ve completed the first part. Click the follow link to access Part II on how we deploy EC2 in our DEV environment.

Part II: https://medium.com/@djakkone/building-a-dev-environment-on-aws-using-terraform-part-ii-31218f39daf7

Thank you for reading and/or following along! Leave us a clap or a comment. Please stay tuned for all my upcoming projects.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment, Share & Follow. Please stay tuned for all my upcoming projects.

— — — — — — -

🛠 Want more? Subscribe to my Newsletter and receive daily AWS cloud updates in your mailbox ➡ My Newsletter

🕒If you haven’t read my earlier articles on AWS services, cloud-native technologies & DevOps tools, be sure to visit my blog and catch up ➡ My Blog

♻ Want more? Discover my comprehensive Cloud tutorials and projects ➡ My Cloud Projects Repo

Contact me directly on LinkedIn or Email.

SUMMARY

There are tons of ways to do this, depending on what kind of computer you’re using. But don’t worry, we’re not going to get lost in the weeds here. Let’s keep it simple and focus on setting one up using Ubuntu.

This setup uses vsftpd (Very Secure FTP Daemon), which is a popular FTP server for Unix-like systems, including Linux, due to its security and performance. It is the default FTP server in the Ubuntu, CentOS, Fedora,

PREREQUESITES

• Access to Linux/Ubuntu
• Knowledge of Bash scripting

STEPS

Step 1: Install vsftpd

• Update your package list and install vsftpd.

• Install vsftp

Step 2: Configure vsftpd

• Open vsftpd configuration file.

• Verify if local user, upload and write commands, FTP connection with SSL/TLS are enabled.

Step 3: Create an FTP User

• Create a user for FTP access. I selected djftp.

• Set a password when prompted and follow the prompts to complete the user setup.

Step 4: Create FTP Directory

• Create a directory for FTP uploads and set the appropriate permissions.

Step 5: Restart vsftpd

• Restart the vsftpd service to apply the changes.

Step 6: Adjust Firewall (if needed)

• Allow FTP traffic through the firewall using`ufw`.

Step 7: Accessing the FTP Server

• Access your new FTP server
• Enter the password when prompted. You should now be able to upload and download files to/from the `/home/djftp/ftp/upload` directory.

ADDITIONAL

Upload a VDI file to the FTP server.

• Navigate to the Upload Directory:

• Switch to Binary Mode (important for non-text files like .VDI).

• Upload the .VDI File

• Verify if VDI file has been uploaded,

Voila! You’ve created a FTP (vsftp) server, and uploaded an Oracle VDI file in the server.

— — — — —

Thank you for reading and/or following along! Leave us a clap or a comment. Please stay tuned for all my upcoming projects.

➡ Don’t forget to catch all my medium Cloud projects.

➡ Seeking lab exercises to replicate? Discover my comprehensive, step-by-step tutorials on GitHub.

➡ Or Contact me directly on LinkedIn or Email.