Add a switch to simplify Terraform debugging

6 min readMar 8, 2024

How to use Azure DevOps capabilities to make it easier for you to analyze Infrastructure as Code Terraform

Introduction

Managing infrastructure using tools like Terraform offers unparalleled flexibility, but it can sometimes be complex to debug. In this article, we will explore an effective method to simplify the debugging process by integrating a switch into your YAML pipelines. More specifically, we will examine how to take advantage of the features offered by Azure DevOps to facilitate the analysis of the execution of deployment processing of your infrastructure via Terraform IaC.

Terraform Debugging Challenges

Debugging Terraform configurations can be tedious, especially in complex environments. Errors can be difficult to identify and resolve, which can lead to deployment delays and operational inefficiencies. With this in mind, adding a switch to your configurations can greatly simplify this process by allowing you to control the execution flow in a more granular way.

Use Azure DevOps for effective analytics

Azure DevOps offers a range of powerful tools for managing the software development lifecycle, but its capabilities also extend to the area of Infrastructure as Code. By integrating your Terraform process with Azure DevOps, you can automate the deployment, validation, and debugging of your infrastructures, providing greater visibility and control over your environment. In the following sections, we will detail the steps to set up this integration and best exploit its benefits to simplify your infrastructure development and management process.

Configuring the setting for Azure DevOps

Azure DevOps offers a host of features to automate and simplify the software and infrastructure development process. One of these powerful features is the ability to configure settings, allowing fine-grained customization of your pipelines and workflows. Parameters in Azure DevOps can be used to store values such as connection strings, API keys, server addresses, etc., providing increased flexibility in managing configurations.

Configuring settings in Azure DevOps is relatively simple. You can set settings at the project level or at the pipeline level, depending on your specific needs. Once the parameters are defined, you can use them in your pipeline jobs, allowing you to separate sensitive values from your code and manage them centrally.

This YAML defines a logging setting for Terraform in an Azure DevOps pipeline:

parameters:
- name: tflog
  displayName: Terraform logs
  type: string
  default: WARN
  values:
  - TRACE
  - DEBUG
  - INFO
  - WARN
  - ERROR

This parameter will allow us to add the following switch in the Azure DevOps interface to the execution of the associated pipeline:

When executing the associated Azure DevOps pipeline, you could therefore choose the level of logs associated with Terraform activities.

Now we will see how to use this parameter so that Terraform activities take this information into account.

Using parameter to set an environment variable

The environment variable TF_LOGis used in the execution context of Terraform code to control the level of detail of the logs generated during execution. It allows you to specify the verbosity level of Terraform logs, which can be extremely useful when debugging or monitoring the execution of Terraform configurations.

This is how it works TF_LOG:

None : No logs are generated. This means that Terraform will not record any output, even if there is an error.
ERROR : Only critical error messages will be logged.
WARN : Only warning messages and errors will be logged.
INFO : General information about the execution will be recorded, in addition to warnings and errors.
DEBUG : Additional debug information will be logged, in addition to INFO, WARN and ERROR levels.
TRACE : The most detailed logs will be recorded, including HTTP requests, Terraform internal state details, etc.

By setting TF_LOGwith one of these levels, you can control the volume and level of detail of logging information displayed while your Terraform code runs. This can be particularly useful for diagnosing issues, understanding execution flow, and monitoring the performance of your infrastructure while minimizing the amount of data recorded when it is not needed.

The parameter defined in the first chapter can be used in the Plan and Apply activities of Terraform via this environment variable definition:

env:
    TF_LOG: $(TF_LOG)

For simplicity, find the complete code for these two activities in the following chapters.

Terraform Plan

    - task: TerraformTaskV4@4
      displayName: 'Terraform Plan'
      env:
          TF_LOG: $(TF_LOG)
      inputs:
        command: 'plan'
        commandOptions: '-out=plan.tfplan -input=false'
        workingDirectory: $(System.DefaultWorkingDirectory)
        environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)

Terraform Apply

      - task: TerraformTaskV4@4
        displayName: 'Terraform Apply'
        env:
          TF_LOG: $(TF_LOG)
        inputs:
          command: 'apply'
          commandOptions: -auto-approve plan.tfplan
          environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)
          workingDirectory: $(Pipeline.Workspace)/tfplan

Run-it !

By choosing a trace WARNI obtain this level of detail when executing my Terraform code:

Link the Terraform log level to the “Enable system diagnostics” option

The “Enable system diagnostics” option in Azure DevOps pipelines is a feature to enable detailed diagnostics logging for a specific pipeline. This feature is useful for debugging issues related to pipeline execution and for obtaining detailed information about performance and possible errors encountered during the build or deployment process.

It is accessible during the execution of the pipeline through a check box:

To link the fact of switching to mode TRACEwhen this is checked you can add these elements to your YAML pipeline:

Use the previous steps Planand Applythen add this block to define a variable TF_LOGin your pipeline:

variables:
- name: TF_LOG
  value: "INFO"

This optional step which, when the box is checked, forces the value toTRACE

- bash: |
         echo "##vso[task.setvariable variable=TF_LOG;]TRACE"
      condition: eq(variables['System.debug'], true)
      displayName: 'If debug, set TF_LOG to TRACE'

If the box is not checked then this step will be skipped:

Otherwise :

And you will then have all the TRACETerraforms:

Complete pipeline

To make the integration simpler, here is the complete pipeline:

parameters:
- name: tflog
  displayName: Terraform logs
  type: string
  default: INFO
  values:
  - TRACE
  - DEBUG
  - INFO
  - WARN
  - ERROR

trigger: none # This pipeline will be triggered by the quality-check pipeline

variables:
- name: TF_LOG
  value: "${{ parameters.tflog }}"
- name: AZ_SERVICE_CONNECTION
  value: "Azure"
- name: AZ_TFVERSION
  value: "1.7.3"

pool:
    vmImage: 'ubuntu-latest'

stages:
- stage: Terraform_Plan
  displayName: 'Terraform Init & Plan'
  jobs:
  - job: Terraform_init_and_Plan
    displayName: 'Terraform Init & Plan'
    steps:

    - bash: |
         echo "##vso[task.setvariable variable=TF_LOG;]TRACE"
      condition: eq(variables['System.debug'], true)
      displayName: 'If debug, set TF_LOG to TRACE'

    - task: TerraformInstaller@0
      displayName: 'Install Terraform v$(AZ_TFVERSION)'
      inputs:
        terraformVersion: $(AZ_TFVERSION)

    - task: TerraformTaskV4@4
      displayName: 'Terraform Init'
      inputs:
        provider: azurerm
        command: init
        workingDirectory: $(System.DefaultWorkingDirectory)
        backendServiceArm: $(AZ_SERVICE_CONNECTION)
        backendAzureRmResourceGroupName: shsprdstorg002
        backendAzureRmStorageAccountName: shsprdstosa002
        backendAzureRmContainerName: tfstate
        backendAzureRmKey: terraform_logs.tfstate

    - task: TerraformTaskV4@4
      displayName: 'Terraform Plan'
      env:
          TF_LOG: $(TF_LOG)
      inputs:
        command: 'plan'
        commandOptions: '-out=plan.tfplan -input=false'
        workingDirectory: $(System.DefaultWorkingDirectory)
        environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)

    - publish: $(System.DefaultWorkingDirectory)
      displayName: Publish Plan Artifact
      artifact: tfplan


- stage: Terrafom_Validation
  jobs:
  - job: ValidationStep
    displayName: Wait for task validation
    pool: server
    timeoutInMinutes: 60 # task times out in 2 hours
    steps:
    - task: ManualValidation@0
      timeoutInMinutes: 60 # task times out in 2 hours

- stage: Terrafom_Apply
  displayName: 'Terraform Apply'
  jobs:
  - job: Terraform_Apply
    displayName: 'Terraform Apply'    
    steps:
      - checkout: none

      - download: current
        artifact: tfplan

      - task: TerraformInstaller@0
        displayName: 'Install Terraform v$(AZ_TFVERSION)'
        inputs:
          terraformVersion: $(AZ_TFVERSION)

      - task: TerraformTaskV4@4
        displayName: 'Terraform Apply'
        env:
          TF_LOG: $(TF_LOG)
        inputs:
          command: 'apply'
          commandOptions: -auto-approve plan.tfplan
          environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)
          workingDirectory: $(Pipeline.Workspace)/tfplan

Conclusion

In this article, we explored an effective approach to simplify the process of debugging Terraform configurations by integrating a switch into your Azure DevOps pipelines. This method provides a convenient way to control the execution flow of your configurations, which can significantly reduce the time needed to identify and resolve errors. By leveraging the capabilities offered by Azure DevOps, you can automate and streamline the analysis of your Infrastructure as Code, improving the efficiency and reliability of your deployments.

Useful links

As usual, here are some links to help you on this subject: