Terraform Deployments With Azure DevOps and Multi-Stage YAML Pipelines

Since my original Terraform Plan articles were published, several new features have been released for AzDO Pipelines, or I’ve discovered features previously unknown to me, and I’ve modified my approach accordingly.

• Switched to Multi-stage pipelines in preparation for running a full Terraform lifecycle (linting/testing, plan, apply) in one pipeline.
• Using Checks / Approvals and Environments which allows us to add aterraform apply stage to our pipeline with an approval gate.
• Using the AzureCLI Task to read in the service principal information for authentication.

If you just want to jump right in and look at the new YAML examples you can find them at Github. Otherwise, read on.

Enable Multi-Stage Pipelines

As of when I’m writing this, multi-stage pipelines are a preview feature. To enable them just follow the steps here.

Once enabled you can take advantage of the new multi-stage UI views.

Checks, Approvals, and Environments

Checks are brand new and are very limited currently. At the moment you can only set a manual approval gate on environments.

Take a look at the documentation page for details. The gist for approval gates is this…

• Create a “Pipeline Environment” in the AzDO UI.
• Edit the Pipeline Environment, click on the ellipses in the upper right of the UI and select Checks.
• Create a new Approval Check and add one or more users and/or groups as approvers.

Previously if I wanted an Approval before a terraform apply would roll out to production, the apply pipeline had to be managed separately in the UI with Release pipelines. Now terraform apply can be managed as it’s own stage within the same pipeline where terraform plan is running.

In your YAML pipeline, you add a deployment job that depends on the Pipeline Environment that has the Approvals configured.

jobs:
- deployment: Apply
  displayName: 'Terraform Apply'
  environment: Production
  pool: My Agent Pool
  workspace:
    clean: all
  strategy:
    runOnce:
      deploy:
        steps:
        - template: Terraform-Apply-Steps.yml

When your pipeline runs, it will pause at this job with a notification in the UI that it’s waiting on an Approver.

Currently, notification options are limited. There’s no built-in ability to send an email or chat notification. Indications are those additional notification options may be coming soon.

I think Microsoft has a lot more planned for the Checks and Approvals so keep an eye on that space.

SPN Authentication using the AzureCLI Task

One of the big changes I’ve made to pipelines recently is switching how SPN authentication works. Previously I was storing the secrets in KeyVault backed Variable Groups, but there were occasional problems with that process. It also felt like an extra layer or two of complexity that I felt didn’t need to be there.

Now instead, I’m using an AzureCLI task to get the SPN information from a configured Service Connection in Azure DevOps.

- task: AzureCLI@1
  displayName: 'Setup Authentication'
  inputs:
    azureSubscription: '$(SUBSCRIPTION_NAME)'
    addSpnToEnvironment: true
    scriptLocation: inlineScript
    failOnStandardError: 'true'
    inlineScript: |   
      echo "##vso[task.setvariable variable=ARM_SUBSCRIPTION_ID]$(az account show --query="id" -o tsv)"
      echo "##vso[task.setvariable variable=ARM_CLIENT_ID]${servicePrincipalId}"
      echo "##vso[task.setvariable variable=ARM_CLIENT_SECRET]${servicePrincipalKey}"
      echo "##vso[task.setvariable variable=ARM_TENANT_ID]$(az account show --query="tenantId" -o tsv)"
      echo "##vso[task.setvariable variable=ARM_ACCESS_KEY]$(az storage account keys list -n ${STORAGE_ACCOUNT} --query="[0].value" -o tsv

azureSubscription tells AzDO which Service Connection to use.

addSpnToEnvironment will add the SPN info to the environment for this script task only.

I use a little AzDO magic in the inlineScript to set those to environment variables that can be used in future script tasks in this job. Since I’m using an Azure Storage Account as my backend I also get and set the ACCESS_KEY for that storage account.

Now in my plan or apply script tasks I can make use of those environment variables as well as variables set in the pipeline or imported from Variable Groups.

- script: |
    terraform init -no-color -input=false
    terraform plan -out=tfplan -no-color -input=false
  displayName: 'Terraform Plan'
  workingDirectory: '$(TFPATH)'
  env:
    TF_IN_AUTOMATION: true
    TF_VAR_subscription_id: $(ARM_SUBSCRIPTION_ID)
    TF_VAR_client_id: $(ARM_CLIENT_ID)
    TF_VAR_client_secret: $(ARM_CLIENT_SECRET)
    TF_VAR_tenant_id: $(ARM_TENANT_ID)
    TF_VAR_vm_username: $(vm_username)
    TF_VAR_vm_password: ${vm_password)

This simplifies the SPN authentication over what I had been doing previously.

Wrapping It Up

You can find the latest YAML examples at the Github.

You can also go back and re-read Part 1 and Part 2 if you’d like.

I’m a big fan of continuous improvement, so drop a comment here or put a PR in the repo if you’ve got ideas.

Finally, a huge shoutout to my company, 10th Magnitude. If you’re looking for a partner with expertise in Azure, CI/CD pipelines, the Hashicorp toolset, Kubernetes, or any of those other cloud and automation buzzwords we can definitely help you. Oh, and if you’re looking for a great place to work we’re hiring too.