Add a switch to simplify Terraform debugging
How to use Azure DevOps capabilities to make it easier for you to analyze Infrastructure as Code Terraform
Introduction
Managing infrastructure using tools like Terraform offers unparalleled flexibility, but it can sometimes be complex to debug. In this article, we will explore an effective method to simplify the debugging process by integrating a switch into your YAML pipelines. More specifically, we will examine how to take advantage of the features offered by Azure DevOps to facilitate the analysis of the execution of deployment processing of your infrastructure via Terraform IaC.
Terraform Debugging Challenges
Debugging Terraform configurations can be tedious, especially in complex environments. Errors can be difficult to identify and resolve, which can lead to deployment delays and operational inefficiencies. With this in mind, adding a switch to your configurations can greatly simplify this process by allowing you to control the execution flow in a more granular way.
Use Azure DevOps for effective analytics
Azure DevOps offers a range of powerful tools for managing the software development lifecycle, but its capabilities also extend to the area of Infrastructure as Code. By integrating your Terraform process with Azure DevOps, you can automate the deployment, validation, and debugging of your infrastructures, providing greater visibility and control over your environment. In the following sections, we will detail the steps to set up this integration and best exploit its benefits to simplify your infrastructure development and management process.
Configuring the setting for Azure DevOps
Azure DevOps offers a host of features to automate and simplify the software and infrastructure development process. One of these powerful features is the ability to configure settings, allowing fine-grained customization of your pipelines and workflows. Parameters in Azure DevOps can be used to store values such as connection strings, API keys, server addresses, etc., providing increased flexibility in managing configurations.
Configuring settings in Azure DevOps is relatively simple. You can set settings at the project level or at the pipeline level, depending on your specific needs. Once the parameters are defined, you can use them in your pipeline jobs, allowing you to separate sensitive values from your code and manage them centrally.
This YAML defines a logging setting for Terraform in an Azure DevOps pipeline:
parameters:
- name: tflog
displayName: Terraform logs
type: string
default: WARN
values:
- TRACE
- DEBUG
- INFO
- WARN
- ERROR
This parameter will allow us to add the following switch in the Azure DevOps interface to the execution of the associated pipeline:
When executing the associated Azure DevOps pipeline, you could therefore choose the level of logs associated with Terraform activities.
Now we will see how to use this parameter so that Terraform activities take this information into account.
Using parameter to set an environment variable
The environment variable TF_LOG
is used in the execution context of Terraform code to control the level of detail of the logs generated during execution. It allows you to specify the verbosity level of Terraform logs, which can be extremely useful when debugging or monitoring the execution of Terraform configurations.
This is how it works TF_LOG
:
- None : No logs are generated. This means that Terraform will not record any output, even if there is an error.
- ERROR : Only critical error messages will be logged.
- WARN : Only warning messages and errors will be logged.
- INFO : General information about the execution will be recorded, in addition to warnings and errors.
- DEBUG : Additional debug information will be logged, in addition to INFO, WARN and ERROR levels.
- TRACE : The most detailed logs will be recorded, including HTTP requests, Terraform internal state details, etc.
By setting TF_LOG
with one of these levels, you can control the volume and level of detail of logging information displayed while your Terraform code runs. This can be particularly useful for diagnosing issues, understanding execution flow, and monitoring the performance of your infrastructure while minimizing the amount of data recorded when it is not needed.
The parameter defined in the first chapter can be used in the Plan and Apply activities of Terraform via this environment variable definition:
env:
TF_LOG: $(TF_LOG)
For simplicity, find the complete code for these two activities in the following chapters.
Terraform Plan
- task: TerraformTaskV4@4
displayName: 'Terraform Plan'
env:
TF_LOG: $(TF_LOG)
inputs:
command: 'plan'
commandOptions: '-out=plan.tfplan -input=false'
workingDirectory: $(System.DefaultWorkingDirectory)
environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)
Terraform Apply
- task: TerraformTaskV4@4
displayName: 'Terraform Apply'
env:
TF_LOG: $(TF_LOG)
inputs:
command: 'apply'
commandOptions: -auto-approve plan.tfplan
environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)
workingDirectory: $(Pipeline.Workspace)/tfplan
Run-it !
By choosing a trace WARN
I obtain this level of detail when executing my Terraform code:
Link the Terraform log level to the “Enable system diagnostics” option
The “Enable system diagnostics” option in Azure DevOps pipelines is a feature to enable detailed diagnostics logging for a specific pipeline. This feature is useful for debugging issues related to pipeline execution and for obtaining detailed information about performance and possible errors encountered during the build or deployment process.
It is accessible during the execution of the pipeline through a check box:
To link the fact of switching to mode TRACE
when this is checked you can add these elements to your YAML pipeline:
- Use the previous steps
Plan
andApply
then add this block to define a variableTF_LOG
in your pipeline:
variables:
- name: TF_LOG
value: "INFO"
- This optional step which, when the box is checked, forces the value to
TRACE
- bash: |
echo "##vso[task.setvariable variable=TF_LOG;]TRACE"
condition: eq(variables['System.debug'], true)
displayName: 'If debug, set TF_LOG to TRACE'
If the box is not checked then this step will be skipped:
Otherwise :
And you will then have all the TRACE
Terraforms:
Complete pipeline
To make the integration simpler, here is the complete pipeline:
parameters:
- name: tflog
displayName: Terraform logs
type: string
default: INFO
values:
- TRACE
- DEBUG
- INFO
- WARN
- ERROR
trigger: none # This pipeline will be triggered by the quality-check pipeline
variables:
- name: TF_LOG
value: "${{ parameters.tflog }}"
- name: AZ_SERVICE_CONNECTION
value: "Azure"
- name: AZ_TFVERSION
value: "1.7.3"
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Terraform_Plan
displayName: 'Terraform Init & Plan'
jobs:
- job: Terraform_init_and_Plan
displayName: 'Terraform Init & Plan'
steps:
- bash: |
echo "##vso[task.setvariable variable=TF_LOG;]TRACE"
condition: eq(variables['System.debug'], true)
displayName: 'If debug, set TF_LOG to TRACE'
- task: TerraformInstaller@0
displayName: 'Install Terraform v$(AZ_TFVERSION)'
inputs:
terraformVersion: $(AZ_TFVERSION)
- task: TerraformTaskV4@4
displayName: 'Terraform Init'
inputs:
provider: azurerm
command: init
workingDirectory: $(System.DefaultWorkingDirectory)
backendServiceArm: $(AZ_SERVICE_CONNECTION)
backendAzureRmResourceGroupName: shsprdstorg002
backendAzureRmStorageAccountName: shsprdstosa002
backendAzureRmContainerName: tfstate
backendAzureRmKey: terraform_logs.tfstate
- task: TerraformTaskV4@4
displayName: 'Terraform Plan'
env:
TF_LOG: $(TF_LOG)
inputs:
command: 'plan'
commandOptions: '-out=plan.tfplan -input=false'
workingDirectory: $(System.DefaultWorkingDirectory)
environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)
- publish: $(System.DefaultWorkingDirectory)
displayName: Publish Plan Artifact
artifact: tfplan
- stage: Terrafom_Validation
jobs:
- job: ValidationStep
displayName: Wait for task validation
pool: server
timeoutInMinutes: 60 # task times out in 2 hours
steps:
- task: ManualValidation@0
timeoutInMinutes: 60 # task times out in 2 hours
- stage: Terrafom_Apply
displayName: 'Terraform Apply'
jobs:
- job: Terraform_Apply
displayName: 'Terraform Apply'
steps:
- checkout: none
- download: current
artifact: tfplan
- task: TerraformInstaller@0
displayName: 'Install Terraform v$(AZ_TFVERSION)'
inputs:
terraformVersion: $(AZ_TFVERSION)
- task: TerraformTaskV4@4
displayName: 'Terraform Apply'
env:
TF_LOG: $(TF_LOG)
inputs:
command: 'apply'
commandOptions: -auto-approve plan.tfplan
environmentServiceNameAzureRM: $(AZ_SERVICE_CONNECTION)
workingDirectory: $(Pipeline.Workspace)/tfplan
Conclusion
In this article, we explored an effective approach to simplify the process of debugging Terraform configurations by integrating a switch into your Azure DevOps pipelines. This method provides a convenient way to control the execution flow of your configurations, which can significantly reduce the time needed to identify and resolve errors. By leveraging the capabilities offered by Azure DevOps, you can automate and streamline the analysis of your Infrastructure as Code, improving the efficiency and reliability of your deployments.
Useful links
As usual, here are some links to help you on this subject: