Using Terraform with Azure — the right way

Nico Meisenzahl
· 5 min read

In this post, I will not write about how to use Terraform to create Azure resources, as there are already tons of guides and hands-on available. You can start here in case you are new to this topic. However, I like to talk about why you should prefer Terraform over other solutions and how to use it in a production-ready manner.

Why Terraform?

The reason why I prefer Terraform over other solutions is that I don’t limit myself to just creating Azure resources and infrastructure. With Azure Kubernetes Service (AKS) for example, I’m not limited to just create the managed Kubernetes cluster itself. Furthermore, I can use the Terraform Kubernetes or Helm provider, to also create needed resources (like RBAC policies, Namespaces, …) within my cluster without the need of another tool. I’m also able to create dependencies like other infrastructure resources (on-premises or multi-cloud) within the same project. With ARM templates for example, which are probably the most common solution for creating Azure resources, I’d have to use multiple tools to get the same outcome.

Thoughts for a production-ready project

While working with Terraform and Azure Cloud in some projects now I would like to share my best practices with you.

The state file

You should store your Terraform state file in a secure and central location. Secure, because it contains sensible data. Central, because you may like to work with your entire team on the same project. Speaking of Azure the best location to store it is in a Blob Storage Account which is supported by Terraform by default. You will need to create the Blob Storage account upfront, which you can easily achieve with the scripts below.

I know this might not be anything new for you but it’s important and therefore needs to be part of this list.

Where to your secrets and IDs

When you are using a Blob Storage Account to store your state file Terraform will need to know the credentials to access it. Regardless of whether you run the CLI manually, as a script or in a pipeline. You shouldn’t store the credentials within your code! It’s better to store them in a secure place and retrieve whenever you need them. Because you are working with Azure the Azure Vault might be the best choice for you.

You will find some useful snippets below how to retrieve your secrets from the Azure Vault to use them with Terraform.

A service account

Depending on your use-case you might like to run Terraform on behalf of a service account instead of your personal account. You can then restrict access rights to those that are only needed for that particular deployment. But, you can also give your service account extended privileges which your personal account might not be allowed to have.

This is achieved by creating a Service Principal, which is then configured in your Terraform configuration. Check out the below script to find out how to create a Service Principal.

Some useful scripts

You are using Terraform because you like the Infrastructure-as-Code principles which allow you to store everything in code. Creating the needed dependencies manually in the Azure Portal is therefore not an option for us!

This script creates a Blob Storage Account to store the state file, a Service Principal which will be used as Terraform service account as well as a Vault to store all needed IDs and secrets:

#!/bin/bashset -e# exports secrets if available; otherwise export $subscriptionId manuallyif [ -f "./creds.sh" ]; then
source ./creds.sh
fi
az account set --subscription $subscriptionId# customize if needed
export rg="tfstate-rg"
export location="West Europe"
export sku="Standard_LRS"
export vaultName="tfstate$RANDOM$RANDOM"
export saName="tfstate$RANDOM$RANDOM"
export scName="tfstate$RANDOM$RANDOM"
# creates a new resource group which will be used for the vault and tf state file
az group create --name "$rg" \
--location "$location" \
--subscription="$subscriptionId"
if test $? -ne 0
then
echo "resource group couldn't be created..."
exit
else
echo "resource group created..."
fi
# creates a vault to store secrets
az keyvault create --name "$vaultName" \
--resource-group $rg \
--location "$location" \
--subscription=$subscriptionId
if test $? -ne 0
then
echo "vault couldn't be created..."
exit
else
echo "vault created..."
fi
# creates storage account used by tf
az storage account create --resource-group $rg \
--name $saName \
--sku $sku \
--encryption-services blob \
--subscription=$subscriptionId
if test $? -ne 0
then
echo "storage account couldn't be created..."
exit
else
echo "storage account created..."
fi
# gets storage account key
export accountKey=$(az storage account keys list --subscription=$subscriptionId --resource-group $rg --account-name $saName --query [0].value -o tsv )
# creats storage container used by tf
az storage container create --name $scName --account-name $saName --account-key $accountKey
if test $? -ne 0
then
echo "storage container couldn't be created..."
exit
else
echo "storage container created..."
fi
# saves secrets to vault
az keyvault secret set --vault-name $vaultName \
--name "sa-key" \
--value "$accountKey"
az keyvault secret set --vault-name $vaultName \
--name "sa-name" \
--value "$saName"
az keyvault secret set --vault-name $vaultName \
--name "sc-name" \
--value "$scName"
if test $? -ne 0
then
echo "secrets couldn't be saved..."
exit
else
echo "secrets are saved in vault..."
fi
# creates a service principal used by tf
export sp=$(az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/$subscriptionId" --years 99 --name tfsp -o tsv)
if test $? -ne 0
then
echo "service principal couldn't be created..."
exit
else
echo "service principal created..."
fi
# gets id and secret
export spSecret=$(echo $sp | awk '{print $4}')
export spId=$(echo $sp | awk '{print $1}')
# save secrets to vault
az keyvault secret set --vault-name $vaultName \
--name "sp-id" \
--value "$spId"
az keyvault secret set --vault-name $vaultName \
--name "sp-secret" \
--value "$spSecret"
if test $? -ne 0
then
echo "secrets couldn't be saved..."
exit
else
echo "secrets are saved in vault..."
fi

You then can use the following snippet to retrieve all needed IDs and secrets when running the Terraform CLI:

...# exports secrets if available; otherwise export $subscriptionId and $tenantId manuallyif [ -f "./creds.sh" ]; then
source ./creds.sh
fi
# customize those if needed
export rg="tfstate-rg"
az account set --subscription $subscriptionId# get vault
export vaultName=$(az keyvault list --subscription=$subscriptionId -g $rg -o tsv | awk '{print $3}')
## extract and export secrets
export saKey=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sa-key -o tsv | awk '{print $5}')
export saName=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sa-name -o tsv | awk '{print $5}')
export scName=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sc-name -o tsv | awk '{print $5}')
export spSecret=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sp-secret -o tsv | awk '{print $5}')
export spId=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sp-id -o tsv | awk '{print $5}')
# export secrets
export ARM_SUBSCRIPTION_ID=$subscriptionId
export ARM_TENANT_ID=$tenentId
export ARM_CLIENT_ID=$spId
export ARM_CLIENT_SECRET=$spSecret
# TF init
terraform init -input=false \
-backend-config="access_key=$saKey" \
-backend-config="storage_account_name=$saName" \
-backend-config="container_name=$scName"
...

You are now ready to run your Terraform project secure and production-ready!

01001101

Stories related to DevOps topics by Nico Meisenzahl. 01001101? First char of my surname.

Nico Meisenzahl

Written by

Senior Consultant @panagenda . @IBMcloud Champion & @Docker Community Leader. Speaker & blogger. 👨‍💻🙋‍♂️ Loves #DevOps, #K8s. His desk is a ping pong table.

01001101

01001101

Stories related to DevOps topics by Nico Meisenzahl. 01001101? First char of my surname.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade