Deploy with Confidence: How Infrastructure Testing Can Help Boost Your Deployment

Deploying code changes to production can be a nerve-wracking experience for any developer. Even with thorough manual testing, it’s impossible to catch all potential issues that could arise from changes to your infrastructure. Infrastructure testing can help alleviate some of this anxiety by automating the testing of your infrastructure, providing a safety net for your deployments.

Integrating Infrastructure Testing in CI pipeline. Diagram inspired from https://devopscounsel.com/wp-content/uploads/2022/07/terratest.drawio-1.png

In this blog, we’ll discuss how to increase the confidence of your deployments by integrating infrastructure testing into your pipeline. Specifically, we’ll be using Terraform to manage our infrastructure, Terratest to write and run automated infrastructure tests, and GitHub Actions to automate the testing process.

We will cover the following topics in this blog post:

1. What are some of the advantages of infrastructure testing?
2. Setting up the environment for GCP
3. Writing Terraform and Terratest code to create a Cloud run application
4. Running the tests locally

What are some of the advantages of infrastructure testing?

Integrating infrastructure testing into your pipeline can provide a number of benefits.

• It provides a controlled environment that allows for precise and reproducible test case execution.
• It increases standardization, repeatability, and consistency of tests performed.
• It provides the platform for automation of tests to reduce human intervention during test execution.
• It offers flexibility in scheduling and executing tests at any time with no manual intervention required.

Operational and Security benefits such as:

• Reduced failures in production.
• Accelerated test execution and early detection of bugs.
• Help in reducing annual operating and business costs.
• Verify that the software operates in an expected and organized manner.
• Improvement in service quality.

Prerequisites

To get started with infrastructure testing, you’ll need to have

1. A GCP account
2. The Google Cloud SDK
3. Terraform
4. Go v1.18.1 & Terratest

Setting up the environment for GCP

First, we need to set up the environment for GCP. Here are the steps to follow:

1. Create a new GCP project.
2. Enable the Cloud Run API, Cloud Build API, and Compute Engine API.
3. Create a service account with the Cloud Run Admin, Cloud Build Editor, and Compute Instance Admin roles.
4. Create a JSON key for the service account and download it.

Writing Terraform and Terratest code to create a Cloud run application

In this blog we will create a simple web application.

The repo has the structure below it consists of a terraform module for cloud run cloud_run_module & examples folder that contains the main infrastructure & test folder that contains our terratest. Finally our CI pipeline that uses Github Actions .github/workflows/ci.yml

terraform-terrates-guide/
├── .github/
│   └── workflows/
│       └── ci.yml
├── cloud_run_module/
│   ├── main.tf
│   ├── outputs.tf
│   └── variables.tf
├── examples/
│   ├── main.tf
│   ├── outputs.tf
│   └── variables.tf
├── test/
│   ├── main_test.go
│   └── terratest.hcl
└── README.md

Terraform Code

Terraform is a popular open-source tool used for managing infrastructure as code. It allows you to define your infrastructure in a declarative language, enabling you to version control and automate the deployment of your infrastructure.

The code mainly is using the cloud run module we created to define cloud run service

module "cloud_run_service" {
  source = "../cloud_run_module"

  project_id   = var.GCP_PROJECT_ID
  region       = var.REGION
  service_name = var.service_name
  image        = var.image
}

Terratest Code

Terratest is a Go-based library used for writing automated tests for infrastructure code. It provides a set of helpers and abstractions to simplify writing tests and offers a simple testing framework for infrastructure code.

func TestCloudRunServiceExample(t *testing.T) {
 t.Parallel()
 gcpProjectID := os.Getenv("GCP_PROJECT_ID")
 region := os.Getenv("REGION")
 service_name := os.Getenv("service_name")
 image := os.Getenv("image")

 terraformOptions := &terraform.Options{
  TerraformDir: "../examples",
  Vars: map[string]interface{}{
   "GCP_PROJECT_ID": gcpProjectID,
   "REGION": region,
   "service_name": service_name,
   "image": image,
  },
 }
 defer terraform.Destroy(t, terraformOptions)
 terraform.InitAndApply(t, terraformOptions)
 serviceURL := terraform.Output(t, terraformOptions, "service_url")    
 time.Sleep(3 * time.Second)
 expectedResponseBody := "Hello, world!\nVersion: 1.0.0\nHostname: localhost"
 http_helper.HttpGetWithRetry(t, serviceURL, nil, 200, expectedResponseBody, 3, 5)

}

terraform-terratest-guide/main_test.go at main · OmarMerghany/terraform-terratest-guide

CI Pipeline

GitHub Actions is a continuous integration and deployment platform offered by GitHub. It allows you to run automated tests and deployment workflows directly in your repository.

name: Test and deploy Terraform app

on:
  push:
    branches:
      - main

env:
  GOOGLE_APPLICATION_CREDENTIALS: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}
  GITHUB_RUN_NUMBER: ${{ github.run_number }}
  GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
  REGION: ${{ secrets.REGION }}

jobs:
  test:
    runs-on: ubuntu-latest
    environment: test

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Install Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: 1.0.4
          terraform_wrapper: false
      
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: '1.18.1'

      - name: Build Terratest
        run: |
          cd test
          go mod init github.com/OmarMerghany/terraform-terratest-guide
          go mod tidy

      - name: Validate Terraform infrastructure
        run: cd examples && terraform init  && terraform validate

      - name: Run Terratest
        run: |
          echo "$GOOGLE_APPLICATION_CREDENTIALS" > key.json
          export GOOGLE_APPLICATION_CREDENTIALS="$(pwd)/key.json"
          export GCP_PROJECT_ID=$GCP_PROJECT_ID
          export REGION=$REGION
          export service_name="test-tf-$GITHUB_RUN_NUMBER"
          export image="gcr.io/google-samples/hello-app:1.0"
          cd test && go test -v -timeout 30m

Running the tests locally

To run the tests, follow these steps:

1. Clone this repository to your local machine: git clone https://github.com/OmarMerghany/terraform-terratest-guide.git
2. Set the environment variables required for the tests:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json
export GCP_PROJECT_ID=<your-gcp-project-id>.
export REGION=<YOUR_REGION>
export service_name=<YOUR_service_name>
export image=<YOUR_IMAGE_NAME> #"gcr.io/google-samples/hello-app:1.0"

1. Navigate to the test directory and run go mod init test followed by go mod tidy
2. Navigate to the test directory and run go test -v

Full Code is available here

GitHub - OmarMerghany/terraform-terratest-guide: This repo includes how to use Terratest to write…

Conclusion:

Infrastructure testing will help catch errors and issues before they make it into production, which can save you time and effort in the long run. It also helps to ensure that changes made to your infrastructure are well-tested and have minimal impact on your users. Finally, infrastructure testing can provide peace of mind and build trust in your deployment process.