Scaffold a production-ready Terraform project on Azure
This post is an updated version of my previous post “Using Terraform with Azure” that I published some time ago. Now, nearly one year later, I have learned a lot and also optimized and extended the examples and code snippets here and there. As a result, we decided to publish all code in this public GitHub repository. This post should provide you with some further details on the project and any details around it. All below code snippets are related to this project.
The project
As mentioned above, we decided to publish everything needed to scaffold a new production-ready Terraform project on Azure. The project is called “Terraform scaffold for Azure” and is available here.
To get started you need to make sure to meet the following requirements:
- a Unix-Shell (we might support Powershell Core in the future, you can use Azure Cloud Shell until then)
- Azure CLI
- Owner access on the target Subscription (to allow Terraform to add future Service Principals as contributors)
If those requirements are met, you only need to execute a single script called up.sh. This will then provision the needed resources:
- a Service Principal used to run Terraform on behalf
- a Storage Container used to store the Terraform state file
- a Key Vault containing all secrets to allow easy and secure access
After executing the script you are ready to create your first Terraform project on Azure. But before that, I would like to give you some details about the project itself.
Where to store the state store
It is best practices to store the Terraform state file in a secure and central location. Secure, because it contains sensible data like secrets. Central, because you may like to work with your entire team on the same project.
Speaking of Azure the best location to store it is in a Storage Container as Blob which is supported by Terraform by default. The needed Storage Container will be created by the up.sh script.
You then need to link the Storage Container within your Terraform project by defining some setting in your main.tf configuration file:
provider "azurerm" {
version = "~> 2.1"
features {}
}
terraform {
backend "azurerm" {
key = "azure.tfstate"
}
}First of all, you need to enable the azurerm Provider to be able to “talk” with Azure. Then you need to define the backend configuration. In the above example, we only define the state file itself and skip to provide any details on how to connect and authenticate with the Storage Container. The reason for this is not to store any secrets in your code. We work around this by using the Terraform CLI, which allows us to specify these details as parameters:
terraform init \
-backend-config="access_key=<your-access-key>" \
-backend-config="storage_account_name=<your-storage-account-name>" \
-backend-config="container_name=<your-storage-container-name>"Why you should use a Service Principal
It is best practice to run Terraform on behalf of a Service Principal. That allows you to either restrict access rights to those needed for your particular deployment. But, you can also give your Service Principal extended privileges that your personal account might not be allowed to have.
The Service Principal will be also be created by the up.sh script. By default, it will be added with owner access rights to allow it to enable future Service Principals which might be created within your Terraform project. This can, of course, be adjusted to your personal needs.
To use a particular Service Principal, we need to make the Terraform CLI aware of it by providing the following environment variables in advance:
export ARM_SUBSCRIPTION_ID=<your-subscription-id>
export ARM_TENANT_ID=<your-tenant-id>
export ARM_CLIENT_ID=<your-service-principal-id>
export ARM_CLIENT_SECRET=<your-service-principal-secret>Secret management
As you may have noticed, the above setup depends heavily on secrets for various authentications. I already mentioned above that you should never store secrets within your code. In order to simplify the use of this setup but still ensure a high level of security, it is recommended to store and retrieve them from a vault.
Since you are working with Azure, we use Azure Key Vault for this. The up.sh script also takes care of this. It will create the Key Vault itself but also directly store all needed secrets in it. With this in place, you can easily retrieve the needed secrets when needed. Regardless of whether you run the CLI manually, as a script or in a CI/CD pipeline.
This is a sample code snippet that you can use to retrieve the secrets locally or as a task within your CI/CD pipeline:
export subscriptionId="00000000-0000-0000-0000-000000000000"
export rg="my-rg"
# sets subscription
az account set --subscription $subscriptionId
# get vault
export vaultName=$(az keyvault list --subscription=$subscriptionId -g $rg --query '[0].{name:name}' -o tsv)
## extracts and exports secrets
export saKey=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sa-key --query value -o tsv)
export saName=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sa-name --query value -o tsv)
export scName=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sc-name --query value -o tsv)
export spSecret=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sp-secret --query value -o tsv)
export spId=$(az keyvault secret show --subscription=$subscriptionId --vault-name="$vaultName" --name sp-id --query value -o tsv)
# exports secrets
export ARM_SUBSCRIPTION_ID=$subscriptionId
export ARM_TENANT_ID=$tenantId
export ARM_CLIENT_ID=$spId
export ARM_CLIENT_SECRET=$spSecret
# runs Terraform init
terraform init -input=false \
-backend-config="access_key=$saKey" \
-backend-config="storage_account_name=$saName" \
-backend-config="container_name=$scName"With this, you are now ready to build your first production-ready and secure Infrastructure-as-code deployment with Terraform on Azure. As mentioned above, all scripts and further details are available here.
