Provision a 3-Tier Network Infrastructure on AWS using Boto3

“Pythonic Provision”

Introduction

In today’s world of cloud computing, the ability to efficiently provision and manage network infrastructure is critical for the success of many projects. As developers or system administrators, we often find ourselves tasked with the responsibility of creating robust and scalable architectures that can handle increasing traffic and application needs.

In this article, I will walk you through how we can utilize Boto3, the AWS SDK for Python, to automate the provisioning of a network infrastructure for a 3-tier architecture application.

Throughout, we will explore how Boto3 enables us to programmatically interact with AWS services, empowering us to build and configure Virtual Private Clouds (VPCs), subnets and more to automate the creation and management of a resilient and scalable network infrastructure on AWS.

Before we dive in, let’s look over the benefits of using Boto3 to provision the networking infrastructure for a 3-tier architecture application.

Benefits

• Flexibility — Boto3 provides the flexibility to create and configure VPCs, subnets, route tables and other networking components programmatically. You can fine-tune the network settings to meet the specific needs of your 3-tier architecture.
• Seamless integration with AWS services — Boto3 seamlessly integrates with various AWS services, allowing you to provision and configure resources giving you the ability to orchestrate the entire infrastructure setup.
• Automation and orchestration — Boto3 allows you to automate the provisioning process. You can write Python scripts that utilize Boto3 APIs to provision resources in a programmatic and repeatable manner to save time and effort while ensuring consistency across environments.
• Customization based on business logic — Boto3 enables you to incorporate custom business logic into the infrastructure provisioning process. For example, you can automatically configure an internet gateway to enable resources to connect to the outside internet. This level of customization helps align the infrastructure with the application’s specific requirements.

Background

Software Development Kit (SDK)

Software Development Kit is a collection of tools, libraries and documentation that developers use to build software applications for a specific platform or service.

Boto3

Boto3 is an AWS provided Python SDK which provides a comprehensive set of APIs and tools that enables developers to interact with and control AWS services programmatically. This allows for easy automation and management of resources and simplifies the integration of AWS services into Python applications.

Three-Tier Architecture

A three-tier architecture is a software design pattern that separates an application into three main tiers or layers, the presentation tier, the application tier also known as the business logic or processing tier and the data tier.

Prerequisites

• AWS Account with an IAM User
• Basic knowledge of the Python programming language
• Basic knowledge and use of an Interactive Development Environment (IDE)
• AWS CLI installed and configured on local machine
• Basic knowledge and use of Linux fundamental commands

Use Case

You’re a Cloud Developer, part of a team working on a web application that follows a 3-tier architecture, consisting of web servers, application servers, and database servers. The team is responsible for automating the deployment and management of the application’s infrastructure on AWS.

It would be ideal to use Terraform or AWS CloudFormation to provision the infrastructure, however the development team doesn't have much experience with infrastructure as code tools, so you decide to use Boto3 to first automate the provisioning of the network infrastructure for the 3-tier application.

Objectives

• Create a custom Virtual Private Network (VPC) with 2 public subnets and 4 private subnets spread across two Availability Zones
• Utilize a NAT Gateway in the public subnet to enable EC2 instances located in private subnets to send traffic to the public and an Internet Gateway for outbound internet traffic.
• Configure a public and private route table.

Step 0

To enhance your understanding and follow this demonstration more effectively, please clone the project’s repository from my GitHub by clicking on the link below. As we move forward, feel free to edit the files as you feel necessary.

GitHub - ifeanyiro9/AWS-3Tier-Architecture-Python: Creating a 3 Tier Architecture on AWS using…

After cloning the repository, you will notice the presence of multiple Python files dedicated to provisioning the network infrastructure.

Here we break up code into different files called modules and import them into the main.py file. This is a common practice in software development to allow for better organization, modularity and reusability of code.

When the code is split into multiple files, each file typically contains related functions or variables that serve a specific purpose or implement a specific feature. This division helps in maintaining code clarity and separation of concerns.

Now, let’s get down and dirty as we break down some Boto3/Python code!

Step 1: Create and configure custom Virtual Private Network (VPC)

Our first objective is to create a custom VPC within our AWS environment, where all the resources for the three-tier architecture will be deployed. To achieve this, we must define a logically isolated CIDR block and carefully create and configure the appropriate subnets and corresponding availability zones (AZ) to deploy them in.

Let’s review the code below of the vpc_subnets.py file to create and configure the custom VPC —

Code Explanation

For all our Python files we will import the Boto3 module, which provides the necessary functionalities to work with AWS services. A low level EC2 client is also created using boto3.client('ec2').

The file includes two functions:

• create_custom_vpc: This function creates a custom VPC (Virtual Private Cloud) in AWS. It also takes three parameters, the cidr_block for the VPC and tags_key and tags_value as the key:value pair for the VPC’s tag. It uses the create_vpc API call from the EC2 client to create the VPC with the provided CIDR block and tags.
• create_custom_subnet: This function creates a custom subnet within a specific VPC. It takes five parameters, az as the desired availability zone for the subnet, the cidr_block for the subnet, the vpc_id which the subnet will be created in and the tags_key and tags_value as the key:value pair for the subnet tag. It uses the create_subnet API call from the EC2 client to create the subnet with the provided parameters and tags.

Both functions return the response from the respective API calls.

Now that we have set up our VPC, let’s move into the next step which focuses on creating the gateways.

Step 2 — Create and Configure Internet and NAT Gateways

Ensuring a secure and efficient connection to the internet is vital when deploying resources within the VPC. To achieve this, one effective approach involves establishing an Internet Gateway and NAT Gateway within a public subnet.

An essential component within the VPC architecture is the Internet Gateway which facilitates communication between instances in the VPC and the internet. On the other hand, a NAT Gateway serves the purpose of enabling instances located in private subnets to establish connections with the internet or other AWS services. Moreover, it offers enhanced security measures for outbound traffic.

Let’s explore the code below of the internet_nat_gateway.py file that creates and configures an Internet Gateway and NAT Gateway —

Code Explanation

The file includes two functions:

• create_attach_internet_gateway: Here we create and attach an internet gateway to our specified custom VPC. The function also takes three parameters, vpc_id and the tags_key and tags_value as the key:value pair for the internet gateway tag. The function uses the create_internet_gateway API call to create the internet gateway with the provided tags and then attaches it to the specified VPC using the attach_internet_gateway API call.
• create_nat_gateway: This function creates a NAT (Network Address Translation) gateway in one of the public subnets in the custom VPC. It takes four parameters, the subnet_id, tags_key_ip and tags_value_ip as the key:value pair for the elastic IP tag and the tags_key_nat and tags_value_nat as the key:value pair for the NAT gateway tag. It uses the allocate_address API call to create an elastic IP address and create_nat_gateway to create the NAT gateway with the allocated elastic IP, with the specified subnet and the provided tags.

Both functions again return the response from the respective API calls.

Building upon our understanding of VPC configuration and network traffic flow, the next step explores key considerations for routing traffic through out the VPC.

Step 3 — Create and Configure Route Tables

At this stage, we have developed Python files to create and configure a customized VPC with required gateways. Now, we focus on effectively managing communication between the services and other deployed components within the VPC. This is where configuring the network traffic flow within the VPC becomes critical. An effective approach to accomplish this is by establishing routing tables to direct the flow of network traffic between subnets.

Let’s review the code in the route_tables.py file below on how we can create and configure the respective route tables —

Code Explanation

The file includes three functions:

• create_pub_rt: This function creates a public route table for the custom VPC and associates it with the internet gateway. It takes five parameters, vpc_id, dest_cidr for the destination CIDR block for the route, int_gw_id for the ID of the internet gateway and the tags_key and tags_value as the key:value pair for the public route table tag. It uses the create_route_table API call to create the route table with the provided tags and VPC ID. This function also creates a route in the route table using the create_route API call to associate the specified destination CIDR block with the internet gateway.
• create_priv_rt: This function creates a private route table for the custom VPC and associates it with the NAT gateway. It also takes five parameters, vpc_id, dest_cidr, nat_gw_id for the ID of the NAT gateway and tags_key and tags_value as the key:value pair for the private route table tag. It also uses the create_route_table API call to create the route table with the provided tags and VPC ID, then it creates a route in the route table using the create_route API call to associate the specified destination CIDR block with the NAT gateway.
• assoc_rt_subnet: In this function, we associate a route table with a specified subnet. It takes two parameters, rt_id for the ID of the route table and subnet_id for the ID of the subnet. It uses the associate_route_table API call to associate the route table with the subnet.

All functions return the response from the respective API calls.

Now that we developed each of Python files needed which will be used as imported modules, let’s transition to the next section to create the main.py Python file.

Step 4 — Develop main.py Python file

The main.py file serves as the entry or central point for executing the program. Main.py imports the other Python files as modules and calls their functions, as well as references their variables. This allows the main.py file to coordinate and control the overall flow of the program by utilizing the functionality provided thtough the imported modules.

By importing other Python files as modules, the main.py file can then invoke their functions, passing any necessary arguments and can manipulate the values stored in variables as needed.

Let’s review the code in the main.py file below and see how we import other files as modules to create a custom VPC with subnets, an internet and NAT gateway and route tables in AWS —

Code Explanation

The code begins by importing boto3 and the required modules, vpc_subnets, internet_nat_gateway and route_tables. A low level EC2 client is also created using boto3.client('ec2').

The code then proceeds with the following steps:

1. Calls the create_custom_vpc function from the vpc_subnets module to create a custom VPC with the specified CIDR block and values of the key:value pair for the tag
2. Retrieves the VPC ID from the response and stores it in the vpc_id variable
3. Calls the create_custom_subnet function from the vpc_subnets module multiple times to create public and private subnets for different availability zones. The function takes parameters such as the availability zone, CIDR block, VPC ID and values of the key:value pair for the tag
4. Retrieves the subnet IDs for the public and private subnets and stores them in respective variables.
5. Calls the create_attach_internet_gateway function from the internet_nat_gateway module to create and attach an internet gateway to the VPC.
6. Retrieves the subnet ID for the NAT gateway and stores it in the subnet_id_ng variable.
7. Calls the create_nat_gateway function from the internet_nat_gateway module to create a NAT gateway using the subnet ID obtained in the previous step.
8. Retrieves the internet gateway ID and NAT gateway ID from the respective responses and stores them in int_gw_id and nat_gw_id variables.
9. Calls the create_pub_rt function from the route_tables module to create a public route table and associate it with the internet gateway.
10. Waits for the NAT gateway to become available by checking its state using the describe_nat_gateways API call.
11. Calls the create_priv_rt function from the route_tables module to create a private route table and associate it with the NAT gateway once the NAT gateway becomes available.
12. Retrieves the public route table ID and stores it in the pub_rt_id variable.
13. Calls the assoc_rt_subnet function from the route_tables module multiple times to associate the public and private subnets with their respective route tables.

The code prints the responses and various IDs during the execution for debugging and verification purposes.

With a solid understanding of the development of the source code to configure the network infrastructure, now let’s execute the Python scripts to automates the provisioning.

Step 4 — Executing code and provisioning network infrastructure

In your desired CLI, navigate to the directory of the cloned repo. Make sure Python 3.0 is installed in your environment, then run the following command to execute the main.py file —

python3 main.py

This will begin to automate the provisioning of the network infrastructure in our AWS environment.

Note, as it starts to execute notice the responses and various IDs of respective API calls for debugging and verification purposes as shown in the image below —

After a period of execution, you will observe a brief pause in the script as it awaits the availability of the NAT gateway. This is accomplished by periodically checking the gateway’s state, as show in the image below—

After the NAT gateways state has been changed to available, the scripts then proceeds as show in the image below —

Now, let’s verify that our networking infrastructure was correctly created in our AWS Management console.

Step 5: Verify the network infrastructure has been provisioned

After the Python scripts has completed its executing we can now proceed to verify that the network infrastructure has been provisioned. We can do this by physically going through the VPC resources in our AWS Management Console.

Proceed by heading to the VPC service in the AWS Management Console, click Your VPC, then select the custom VPC created.

Scroll down, then click Resource Map and to see all the network resources mapped by lines to represent the relationships as seen below —

If you click show details you'll be able to see more details on the resources. Additionally, if you hover the mouse over each resource you’ll see highlighted the direct relationship of that resource to other resources within the custom VPC.

Congratulations!

You’ve successfully completed “Pythonic Provision”. We’ve utilized the power of Boto3, the AWS SDK for Python, to automate the provisioning of a network infrastructure for a 3-tier architecture application!

Clean Up

The disadvantage of using Boto3 to provision infrastructure over a infrastructure as code tool like Terraform, is that we don’t have the capability Terraform offers of using one command to automatically deprovision every resource that was deployed.

With Boto3 we’ll have to either manually delete each resources or create a separate script to delete/remove all the recourses.

You are free to create a separate script to delete/remove all the recourses, however, for this demonstration, follow the following steps in order to manually delete the network infrastructure resources provisioned from the Python scripts especially so we don’t incur any unnecessary costs —

1. Delete the NAT Gateway
2. Release the Elastic IP Address
3. Delete custom VPC

Ifeanyi Otuonye is a 6X AWS Certified Cloud Engineer skilled in DevOps, Technical Writing and instructional expertise as a Technical Instructor. He is motivated by his eagerness to learn and develop and thrives in collaborative environments. Before transitioning to the Cloud, he spend six years as a Professional Track and Field athlete.

In the early 2022, he strategically embarked on an mission to be a Cloud/DevOps Engineer through self study and joining the 6 month accelerated Cloud program, Level Up In Tech !

In May 2023, he accomplished that goal and landed his first Cloud Engineering role and has now set another personal mission to empower other individuals on their journey to the Cloud.

REXTECH Community

If you’ve got this far, thanks for reading! We hope it was worthwhile to you.

If it was, please show your appreciation by giving this article as many likes as it deserves and following it’s author.

Don’t forget to follow us on Medium, LinkedIn and X (Twitter)!