Deployment Of A Reusable Saas Multi-tenant AWS Infrastructure Using Terraform Modules Securely Storing Terraform Configuration Files On AWS Code Commit.
In this project based , I designed and deployed a reusable, multi-tenant SaaS infrastructure on AWS using Terraform modules and securely store the terraform configuration files on AWS CodeCommit. The goal of this project was to provide a seamless, efficient, cost-effective and scalable method for deploying and managing SaaS applications on AWS. I will be implementing best practices and focus on creating the essential components to achieve the desired outcome.
The infrastructure was based on AWS services such as EC2 instances, DynamoDB databases, and S3 buckets. To ensure the secure storage of Terraform configuration files, I used AWS CodeCommit.
Infrastructure as Code — IAC: Infrastructure as code provisions the infrastructure in cloud or on-premises writing code in a declarative way usually in JSON Format. Using IAC scripts and tools to automate the provisioning of the infrastructure by leveraging APIs provided by different cloud providers APIs. Popular infrastructure as code provisioning tools includes Terraform, AWS CloudFormation, Azure Resource Manager, GCP Deployment Manager, and OCI Resource Manager.
Terraform: Terraform is an Infrastructure as Code IAC tool provided by HashiCorp in a declarative JSON Format called HashiCorp Configuration Language. It lets you write Infrastructure provisioning and deployment as code that builds, change, and maintain source version control both on Cloud and on-premises resources. Terraform is Cloud agnostic, Declarative language, Immutable Infrastructure, Agentless & Masterless, HCL Language in JSON format and Open-Source free version. Terraform fundamentals consist of working with different cloud providers, variables & outputs, resource attributes & dependencies. Its declarative language with infrastructure resources is “Block of Code” with different types of blocks like locals, Terraform, Providers, Resources, Data, Module, Variable, and Output. Terraform consists of an ‘init’ file that initializes the terraform executable version installed in the local working directory. Its basic code is separated into different files like ‘main.tf’, ‘resources.tf’, ‘variables.tf’, and ‘output.tf’ define to provision infrastructure and deploy them by using terraform apply command.
It’s always good practice to use terraform provider documentation whenever we are creating terraform configuration files as the cloud providers continuously update their provider’s resource versions information.
Terraform input variables can be defined in a terraform.tfvars.tf file, these variables can be referred to in a resource.tf file when defining the cloud provider resources like AWS instance Amazon machine image-ami and instance type. Using variables in one file and referring them in the resource file gives an option to change the variable default values in one place and the changes are reflected in all places in the resource file where ever these variables are used. Terraform can take the variables from command line input like asking for default variable value, export the value using TF_VAR_’variablename’, using -var=”variablename=variablevalue”, using terraform variable file and providing variable details in the file, and the last one is by explicitly specifying the terraform variables file.
Terraform output values were the result of the resources being created in the resource file. These output values can be defined in an ‘outputs.tf’ file which will give the results by displaying provisioning resource details created for illustration, the outputs file can output the results of an AWS instance web server public Ip address.
Terraform reference attributes will be dependent on the resources created attributes and these attributes which are dependent on other resource attributes can be implicitly handled by Terraform or can be explicitly mentioned in the order by using the depends_on attribute name. In this way terraform gives the flexibility of creating the resources in a more sequence dependent order.
Terraform State, Once the terraform initiates and plans the resources to be provisioned then we run the terraform apply command to deploy the resources. For the first time when apply command is executed then the terraform creates a file called terraform state file which stores all the information about the resources created, their attributes, and the state of the resources that are deployed. From the second time onwards when we use the apply command, terraform uses this state file to determine whether to create, update or delete the resources and also it creates terraform.tfstate to store the current state and the previous state is stored in terraform.tfstate.backup. This terraform.tfstate file can be stored locally or remotely as it contains sensitive information in the file and the best practice is to store this file remotely so that only one person in the team can work on this file and lock the file to all the other users like DevOps Engineers.
Terraform workflow consists of these steps,
· Writing Infrastructure configuration code in terraform.tf and resources.tf files
· Initialize terraform under the working directory download files, check the configuration, provider, and modules — Command used: terraform.init
· Display the plan of the resources to be created to review before applying — Command used: terraform plan
· Applying by provisioning and deploying resources and updating desired state — Command used: terraform apply
Terraform SubCommands, Terraform consists of many sub-commands, and some of the frequently used ones are terraform fmt, validate, show, providers, output, console, and destroy.
Terraform Modules, Modules provide to scale the creation of resources into separate files depending on the resource types like EC2 Instances, networks or databases as child modules and all these child modules can be referred from the main terraform file which is the Root module. In this way modules will help to reduce the code and reuse the same child module as needed from the Root module, for illustration, if we want to create 20 EC2 Instances then we will define EC2 Instance resource file ones as child module and invoke this child module from root module with a count equal to 20 to create 20 EC2 Instances. These modules can be local or remote.
Terraform Provisioners, Provisioners allow to run specific tasks or actions after the resources are created from local or remote machines. They help to know where the EC2 Instance is created like region, location, and know the Amazon resource name-arn and about the tags used for resources.
Terraform modules allows one to group one, to multiple resources, so you can break your infrastructure code to reusable and configurable portions of code, where you can enforce certain defaults
The key tasks of this project includes:
- Connect to the new AWS CodeCommit repository (humangov-infrastructure) to securely store Terraform configuration files.
- Designing a Terraform module that creates and configures AWS resources such as EC2 instances, RDS databases, and S3 buckets for multi-tenants SaaS applications.
- Implementing AWS IAM Roles and policies to provide secure access to the deployed resources
- Configuring the terraform backend to use an S3 bucket for storing the Terraform state files
- Deploying the infrastructure using Terraform and testing the setup to ensure it is working as aspected
- Documenting the process and providing instruction on how to re-use and customize the terraform module for different tenants.
Implementation
Phase 1- Terraform Modules
In cloud9, I am going to navigate to the human-gov-infrastructure folder created in my last project and create a root directory folder called terraform to store terraform configuration files. I will create a module directory inside the terraform folder. This module will deploy several resources in each state. The module consist of Amazon EC2 instance, Amazon Dynamo DB and Amazon S3 all of which are still under human-gov-infrastructure which is connected to AWS CodeCommit remote private repository “human-gov-infrastructure”.
The folder structure is created as follows:
i) I first created state-specific variables under in variables.tf file.
ii) In the main.tf file, I created all the resources like AWS security group, AWS EC2 Instance, AWS DynamoDB, random string, and Amazon S3 bucket. I created a security key and asscoiated the security key with the ec2 instance.
iii) The outputs.tf file displayed the values of the resources after they were created like AWS EC2 instance public DNS, AWS DynamDB table name, Amazon S3 bucket name.
I created three more files variables.tf, main.tf, and outputs.tf outside of the modules folder in the terraform root directory. The variables.tf file which holds the list of states where the infrastructure needs to be provisioned.
main.tf file mentions about the AWS provider with the region and defines the root module to call the child module under the modules folder for each state in the list so that providing the infrastructure for each state with AWS security group, AWS EC2 Instance, AWS DynamoDB, random string, and Amazon S3 bucket.
Outputs.tf file displays the details of each state EC2 Instance public DNS, AWS DynamoDB table name, and Amazon S3 bucket name.
I uploaded the private key humangov-ec2-key.pem created earlier to Cloud9 for saving purposes.
I then applied the terraform configurations. I first formated all the configuration files of child and root modules and then I ran terraform init to initialize terraform and validate them. Next step was to run terraform plan and then terraform deployed 15 resources. Five resources for each state in the child module like AWS security group, AWS EC2 Instance, AWS DynamoDB, random_string, Amazon S3 bucket with state prefix used for all these deployments.
The terraform configuration commands are:
terraform fmt — to format and configure codes
terraform init- to initialize terraform
terraform validate- helps to spot errors i.e synthax errors
terraform plan- shows what terraform is planning to do
terraform apply- carry out action
The reosurces have been successfully creates and I confirmed by checking AWS.
EC2 Instances for each state
S3 buckets for each state
DynamoDB Tables
We can clearly see that everything ran successfully. After implementation and everything working successfully remember to destroy all the resources using terraform destroy.
Phase 2- Placing Terraform State file in the remote backend
When we use terraform apply then the terraform will create a terraform state file for the first time and from the second time onwards it will have the current terraform state file and previous state information created in the new file terraform state backup file. These state file needs to be maintained and stored in a remote place using the Amazon S3 bucket as this state file might contain sensitive information like passwords for example.
Now the terraform state is locally available in the Cloud9 development environment of the Devops Engineer at HumanGov. This setup is risky and the state file should not be maintained locally because if this Cloud9 is deleted or the crashes then this state file will be deleted. I also need to enforce a lock after moving the state file such that 2 users should not be able to make changes at the same time and apply changes to infrastructure and for this reason, terraform lock file should be saved in the backend. For this case let’s save the Terraform state file in S3 bucket backend and Terraform lock file in DynamoDB table. We will create an Amazon S3 bucket and DynamoDB table using AWS CLI and not through Terraform as we don’t want to delete these 2 backend resources accidentally using Terraform destroy.
In myroot project directory, I created a backend.tf file in myproject directory:
touch backend.tfEdit the backend.tf file with the following content. Replace your-unique-state-bucket-name with the name of the bucket you created in step 4:
terraform {
backend "s3" {
bucket = "humangov-terraform-state-2500"
key = "terraform.tfstate"
region = "us-east-1"
encrypt = true
dynamodb_table = "humangov-terraform-state-lock-table"
}Initialize Terraform:
terraform init
Now terraform is storing the state file remotely in the s3 bucket. I verified this by checking the contents of the s3 bucket using the AWS CLI.
If we make changes to any of the configuration files like variables.tf file and edit the file to apply infrastructure to only one state, then we will notice that the terraform.tfstate file is not updated locally anymore. It is updated remotely at the backend in the S3 bucket ‘humangov-terraform-state-2500’ and the terraform.lock file is always saved remotely as an item in DynamoDb table ‘humangov-terraform-state-lock-table’ locking the infrastructure changes to be applied at any point of time by only one DevOps Engineer.
Phase 3- Moved Terraform Configuration Files to AWS Code Commit
To maintain the source code version control of terraform configuration files I securely stored all these files on the remote private AWS Code Commit repository ‘human-gov-infrastructure’. In this way, all the DevOps Engineers can work in collaboration with the same set of files by creating their own local working directory and cloning this remote repository ‘human-gov-infrastructure’ and using pull and push to the master branch.
If I make changes to any of the configuration files then the local terraform.tfstate file is not modified as we defined to be updated at the remote backend location in the S3 bucket ‘humangov-terraform-state-2500’.
When I am pushing these terraform files to the remote private repository ‘human-gov-infrastructure’ of AWS Code Commit, I need to exclude all the files that are used to initiate the terraform executable files that are stored locally in the working directory of the DevOps Engineer local development environment. I am going to to create a file called ‘.gitignore’ and add all the file extensions to ‘gitignore’ file to exclude the files being added to the remote repository.
I will first add all the files to the git repository and then push it to the remote repository. I add all the files from the local repository to the staging area first by using the ‘git add .’ command and then use git status to check that the files have been successfully added to the staging area. From the staging area I will add the files to the git repository using ‘git commit’ command with a message. From the git repository I will push the files to remote private repository ‘human-gov-infrastructure’ of AWS Code Commit.
The files have been added successfully to remote private repository ‘human-gov-infrastructure’ of AWS Code Commit.
I have successfully established a reusable, scalable, and secure SaaS multi-tenant infrastructure on AWS. To achieve this goal I worked as the cloud devops engineer using terraform infrastructure as code (IAC) configuration files and terraform modules to provision infrastructures in scalable, quick and cost effective manner. I also moved the configuration files to remote private repsoitory provided by AWS Code Commit for better security and to allow collaborative work between DevOps Engineers.
