Manage large infrastructure at scale with the power of Terragrunt and GitHub Action

What would you do in a situation of building and maintaining a large infrastructure across multi-clouds, multiple accounts, and multiple regions? Don’t let it feel like an overwhelming task. With the right approach and tools, it can be done with ease and efficiency. By embracing the principles of Infrastructure as Code (IaC), you can automate the process of provisioning, configuring, and managing your infrastructure, ensuring consistency and reducing the risk of manual errors.

To achieve it, make use of Terragrunt, a powerful tool that automates the creation, modification, and versioning of cloud resources. Integrating Terragrunt into your CI/CD process through Github Actions can further increase efficiency and ensure consistent deployments.

In this blog, we’ll take a look at some important concepts of Terragrunt, and show you how to automate Terragrunt CI/CD in GitHub Actions, making the maintenance and management of your infrastructure easy.

But first, let’s establish what IaC is

Infrastructure as Code (IaC) is a methodology for efficiently and consistently handling and establishing infrastructure. With IaC, configuration files are created that contain your infrastructure specifications, which makes it easier to edit and distribute configurations.

What is Terragrunt?

Terragrunt is a thin wrapper tool for Infrastructure as Code (IaC) that provides extra functionality on top of Terraform, such as keeping your Terraform code organized and managing shared components. It also helps to reduce duplication and improve maintainability by sharing and reusing code.

Before diving into how to automate Terragrunt CI/CD in GitHub Actions, it’s important to understand the concepts that make Terragrunt such a valuable tool:

1. DRY (Don’t Repeat Yourself) principle — Terragrunt supports the DRY principle by allowing you to share configuration and variables between different environments, projects, and teams. It also enables you to define reusable modules and configurations, reducing duplication of code and improving maintainability.
2. Terragrunt structure — The simplicity of the Terragrunt structure makes it easier to maintain and scale over time, ensuring that it is consistent and reusable. This is done by separating Terraform modules into their own directories and using a consistent structure for configuration.
   Later, I will demonstrate how to utilize this folder structure in your CI/CD pipeline.

3. Simple state leads to quick execution — As long as you have a simple and well-defined structure, you ensure that your Terraform state is organized, short, and simple. As a result of the clear structure, you can enjoy short runs that improve your workflow and save time. And this is a crucial and significant consideration.

4. Improved versioning — Terragrunt integrates with version control systems, making it easier to manage changes to your infrastructure and ensure consistency between environments.

With this background in mind, let’s dive into the process of automating Terragrunt CI/CD in GitHub Actions! This process involves creating a workflow in GitHub Actions that runs Terragrunt as part of your CI/CD pipeline. This will ensure that your Terraform code is tested and deployed automatically, reducing the risk of errors and ensuring that your resources are up-to-date.

Let’s wrap it with GitHub action!

GitHub Actions provides a simple and efficient way to automate all infrastructure as code development processes. So what do we mainly gain when we implement Terragrunt CI/CD in GitHub Actions?

• GitOps — Using Git as the single source of truth for infrastructure as code is a powerful way to manage infrastructure in a collaborative and efficient manner.
• Audit — Changes to your infrastructure are so sensitive, and need to be recorded and audited. This makes it easier to track changes and ensure that your infrastructure is always up to date.
• Self-service — Another important advantage of having a CICD system for IaC is the ability to allow self-service for developers. With CICD, all infrastructure as code management processes are automated and managed within the GitHub pull requests lifecycle, which makes it easier for developers to manage their own infrastructure.

GitHub Action flow diagram

1. Trigger — The workflow is triggered by the event of a pull request being made against the master branch. When a PR opens, a GitHub Action workflow for the Terragrunt plan is triggered and runs automatically.

2. Get changed files — This step utilizes the folder structure we saw before, in order to run the plan execution only on files that have been modified in the pull request. This is a crucial strategy that saves a lot of time!
So if a DevOps engineer wants to add or change a specific resource, the GitHub Action workflow first identifies which files have been changed/added/modified in the PR, and then runs the plan only for those resources that have been changed.

3. Terragrunt validation — This phase executes Terragrunt fmt and validation to check if the code is properly formatted and valid. If the code is not valid, the step will fail, resulting in the failure of the plan job.

4. Init — Run Terragrunt init to initialize a working directory containing Terraform configuration files.

5. Plan — This step runs the actual plan only for those files that have been changed.

6. Publish plan output — Update pull request with the plan outpu

7. Terragrunt plan status — The success of the Terragrunt plan status step is necessary for merging the PR, as without it, the apply cannot be executed.

8. Apply — After the plan finishes successfully, the developer who opened the PR can review the planned changes and see if they got the expected results. Here, we have a great self-service solution where the person who opened the pull request has control over the apply, and is not dependent on anyone else. In our CICD flow, merging a PR means Terragrunt apply execution. This is highly sensitive and maintained by 3 mandatory requirements:

• Require branches to be up to date before merging, ensuring that a pull request includes all the latest commits from the default branch. Skipping this requirement can result in an unintentional change of resources created in recent commits.
• The plan status must complete successfully.
• Require at least 1 code review approval to minimize the potential for human error.

9. Notification — Send a notification in Slack to the person who opened the PR with details on whether the apply was successful or failed with a link to the workflow.

In conclusion, automating Terragrunt CI/CD in GitHub Actions is a great way to improve your Terraform workflow and ensure consistent and efficient deployments. By leveraging Terragrunt’s additional functionality and incorporating it into your CI/CD process, you can take your Terraform management to the next level.

So managing large infrastructure at scale doesn’t seem like an overwhelming task anymore, does it?