How to Build a CI/CD pipeline for Terraform infrastructure on Google Cloud Platform (GCP)

CI/CD pipeline for Terraform infrastructure

cypik.com

Building a CI/CD pipeline for Terraform infrastructure entails automating the provisioning and management of infrastructure with Terraform, as well as integrating continuous integration and continuous delivery practices. The following are the general steps for establishing a CI/CD pipeline for Terraform infrastructure:

1. Version Control System (VCS): Start by creating a repository in a version control system like Git, for example GitHub, GitLab, Bitbucket This will be the central location to store your Terraform configuration.

2. CI/CD Platform: Choose a CI/CD platform that integrates well with your VCS and GCP. Popular options include Jenkins, GitLab CI/CD, CircleCI, or GitHub Actions.

Set up a new project or repository on your chosen CI/CD platform.

3. Service Account: Create a service account in GCP with appropriate permissions to manage resources in your target GCP projects.

Create a JSON key for this service account and store it securely, as it will be used by your CI/CD pipeline to authenticate with the Google Cloud Platform.

4. Define the workflow: We need to define a workflow for our CI/CD pipeline. We will trigger the pipeline on every push to the master branch of our repository.

name: Terraform CI/CD

on:
  push:
    branches:
      - master

5. Set up the environment: We will configure the environment for our pipeline. This includes the installation of Terraform and gCloud CLI.

jobs:
  terraform_apply:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          terraform_version: 1.0.0

      - name: Install the gcloud CLI
        uses: google-github-actions/setup-gcloud@v2
        with:
          project_id: ${{ secrets.GOOGLE_PROJECT }}
          service_account_key: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}
          export_default_credentials: true
          version: '>= 363.0.0'

6. Authenticate with GCP: We will authenticate to the Google Cloud Platform using the service account credentials provided.

      - name: Authenticate with Google Cloud Platform
        uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}

7. Configure the Terraform Backend: Replace "your-bucket-name" it "your-prefix" with your actual GCS bucket name and prefix.

- name: Configure Terraform Backend
  run: |
    cat <<EOF >backend.tf
    terraform {
      backend "gcs" {
        # Add your GCS backend configuration here
        bucket  = "your-bucket-name"
        prefix  = "your-prefix"
      }
    }
    EOF

- name: Terraform Init
  run: terraform init

8. Initialize Terraform: We will start Terraform in your working directory.

      - name: Terraform Init
        run: terraform init

9. Plan Infrastructure Changes: Next, we’ll generate an execution plan for our Terraform changes.

      - name: Terraform Plan
        run: terraform plan -out=tfplan

10. Apply infrastructure changes: Finally, we’ll apply the Terraform changes if the push event is on the master branch.

      - name: Terraform Apply
        if: github.ref == 'refs/heads/master' && github.event_name == 'push'
        run: terraform apply -auto-approve tfplan

11. Terraform Destroy: destroy infrastructure if explicitly triggered, ensuring cleanup and resource management.

- name: Terraform Destroy
        run: terraform destroy -auto-approve
        env:
           credentials_json: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}

Example: GitHub Actions Workflow

name: Terraform CI/CD

on:
  push:
    branches:
      - master

jobs:
  terraform_apply:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          terraform_version: 1.0.0

      - name: Install the gcloud CLI
        uses: google-github-actions/setup-gcloud@v2
        with:
          project_id: ${{ secrets.GOOGLE_PROJECT }}
          service_account_key: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}
          export_default_credentials: true
          version: '>= 363.0.0'

      - name: Use gcloud CLI
        run: gcloud info

      - name: Authenticate with Google Cloud Platform
        uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}

      - name: Configure Terraform Backend
        run: |
          cat <<EOF >backend.tf
          terraform {
            backend "gcs" {
              # Add your GCS backend configuration here
              bucket  = "your-bucket-name"
              prefix  = "Terraform"
            }
          }
          EOF

      - name: Terraform Init
        run: terraform init


      - name: Terraform Init
        run: terraform init

      - name: Terraform Plan
        run: terraform plan -out=tfplan

      - name: Terraform Apply
        if: github.ref == 'refs/heads/master' && github.event_name == 'push'
        run: terraform apply -auto-approve tfplan

      - name: Terraform Destroy
        run: terraform destroy -auto-approve
        env:
           credentials_json: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}

Important Considerations

1. State Management:

• Utilize Google Cloud Storage (GCS) as the remote backend for Terraform state storage.
• Configure versioning for the GCS bucket to track changes over time.
• Encrypt the state file using Google Cloud KMS for added security.
• Ensure proper IAM permissions are set for accessing the GCS bucket.

2. Infrastructure as Code (IaC):

• Organize Terraform code into reusable modules that encapsulate specific functionalities, such as network configurations, compute resources, and storage setups.
• Leverage Terraform workspaces to manage multiple environments (e.g., development, staging, and production) within the same configuration.
• Use variables and data sources to make the Terraform code modular and adaptable to different environments.

3. Security:

• Store sensitive information, such as service account credentials and API keys, in Google Cloud Secret Manager or Google Cloud KMS.
• Implement Google Cloud IAM roles and policies to control access to GCP resources.
• Utilize Terraform’ssensitive attribute to mark sensitive data within the code, preventing accidental exposure.
• Regularly review the Terraform code for security vulnerabilities and compliance with best practices.

4. Rollback Mechanism:

• Implement a strategy for rolling back changes in case of failures or issues during the Terraform deployment.
• Utilize Terraform’s state history and versioning features to track changes and revert to a previous state if necessary.
• Integrate monitoring and alerting systems to detect failures early and trigger rollback processes automatically if needed.

Adapting the pipeline to GCP-specific tools and requirements involves integrating Terraform with Google Cloud Build or other CI/CD platforms.

You can create a robust and secure Terraform deployment pipeline tailored to your project’s requirements on the Google Cloud Platform.

This GitHub Actions workflow automates the Continuous Integration/Continuous Deployment (CI/CD) process for managing infrastructure with Terraform. The workflow is triggered on every push to the master branch of the repository. It leverages Google Cloud Platform (GCP) services for infrastructure provisioning and management.

Enjoy it! 🍻 That’s It; we are done...

For seamless Cloud Management incorporating DevOps as the core of the methodology, reach out to us at info@cypik.com

Cypik

About the author:

My name is Suresh Yadav, and I am an experienced Linux enthusiast and DevOps engineer. I’m passionate about automating and streamlining development processes, and currently, I work as a DevOps Engineer at Cypik. I specialize in cloud technologies, with a focus on the Google Cloud Platform (GCP).