Automating AWS Infrastructure with Terraform: A Modular Approach for Efficiency and Scalability
Hello, dear Medium readers. I hope you are doing well during this final period of 2023. This will likely be my last post for the year. In 2023, I embarked on my writing journey here on Medium, and it has brought me immense joy. I’ve always had a passion for writing, but until now, I hadn’t come across a suitable opportunity to indulge in it. So, I extend my special thanks to all of you.
Task of the day
Now, let’s delve into the latest implementation I’ve undertaken, utilizing Terraform. In this project, grounded in a real-world scenario, I conceptualized and deployed a reusable, multi-tenant SaaS infrastructure on AWS, leveraging Terraform modules. I’ve previously discussed these modules in detail in a couple of posts, namely this one and this one. The infrastructure relied on AWS services like EC2 instances, DynamoDB databases, and S3 buckets.
In this deployment, the hypothetical company HumanGov, a Software as a Service provider serving government organizations, has solicited an application tailored to address the specific requirements of the US Government’s business rules. Typically, these business rules are non-negotiable and often lead to an escalation in the application’s costs. However, omitting them is not a viable option for delivering a satisfactory solution.
In this specific scenario, it is imperative to deploy a distinct set of services (EC2, DynamoDB, and S3 Bucket) for each state in the US. The mandate for separate application instances stems from the intricacies of the business rules. Each application must operate independently to adhere to these rules.
Services used
As depicted in the image below, the US Government’s directive was to establish isolated applications. Deploying a single VM EC2, a DynamoDB table, and the creation of an S3 Bucket typically takes between 5 and 10 minutes — not a significant challenge, right? However, the complication arises when considering the scale of the country, especially in the case of the USA.
As you may be aware, the USA comprises 50 states. In practical terms, deploying all of these elements can take anywhere from 250 to 500 minutes — a process that is neither scalable nor automated. To address this challenge, Terraform emerges as the ideal tool for automation, providing scalability. As highlighted in previous posts, Terraform excels at deploying and dismantling infrastructure in Cloud Service Providers through code.
To streamline this process, I’ve also incorporated additional services beyond the essential requirements, aiming to expedite tasks and enhance productivity. The first is AWS Cloud9, as previously introduced here, eliminating the need for authentication methods and streamlining the workflow. When utilizing the AWS IDE in the AWS environment, authentication becomes seamless, relying on the permissions associated with your user or the User Group to which your user belongs. The second service is AWS CodeCommit, AWS’s Git service, comparable to Microsoft’s GitHub.
IaC — Terraform
The Terraform tool, developed by HashiCorp, operates on a declarative programming language. The significance of this lies in the fact that the code you generate not only deploys infrastructure but also allows for automation within your code, akin to languages such as Python, Java, C, and more. The goal is to adopt a modular approach to code, as emphasized in the best practices outlined in the book “Clean Code: A Handbook of Agile Software Craftsmanship.”
As depicted in the image below, I’ve implemented a modularized structure to reduce the need for extensive code writing, thereby enhancing scalability. While I won’t delve deeper into the explanation of Terraform Modules, as I’m still navigating the learning curve, you can refer to this link from a previous post on Medium that provides insights into Terraforms Module.
One last crucial point: in the Proposed Architecture Image, on the lower-left section, it illustrates that the Terraform backend file, the remote state, will be stored in DynamoDB — a highly available database — and the historical data will be stored in an S3 Bucket. Following HashiCorp’s recommendation, the remote state should be stored in the cloud to prevent multiple individuals from deploying changes to the infrastructure simultaneously, especially when someone is actively working on it.
Conclusion
In summary, the implementation of a task in AWS involves leveraging Terraform to automate and scale infrastructure deployment. The use of a modularized code structure streamlines the process, aligning with the principles of clean code.
The proposed architecture includes storing the Terraform backend file in DynamoDB for remote state and utilizing an S3 Bucket for historical data, ensuring high availability. This approach, in line with HashiCorp’s recommendation, prevents concurrent infrastructure changes, promoting a seamless and efficient workflow.
