How to deploy Azure Key Vaults with Terraform

Published in

Globant

8 min readJul 8, 2022

Introduction

When we are developing our applications it is important to take special care of the way we guard our secrets and how we control the access to them. From an unintentional mistake till a malicious attack, there are many reasons why we need to manage our secrets correctly.

If you are using Azure stack technology, Azure Key Vault is a great cloud service to guard your secrets. It allows you to store secrets like passwords, certificates or cryptographic keys, and you can manage who or what services can access your secrets and you can control which types of permissions they will have.

In large projects, you will probably work with different products and every product will work with different environments like DEV, TST, UAT and PROD. Therefore you will need a key vault per application and per environment if you follow the best practices. This will allow you to manage your secrets properly and it also will reduce the impact in case of a breach.

If you have a small amount of projects they are relatively easy to manage, but what happens when you have large projects? Fortunately, today we have different tools and services that can help us. Today, I will talk about Terraform and how we can use it to manage our key vaults.

Quoting the Terraform official site: “HashiCorp Terraform is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share.” Basically it helps you configure your infrastructure as code in a more efficient way. It keeps track of your real infrastructure in a state file, which acts as a source of truth for your environment. Then, you create and configure your terraform files with the changes you want to make to your real infrastructure.

Considering the above, I will share the scripts I use to maintain our infrastructure for creating the necessary key vaults, assigning the permissions and generating the service connections to integrate it with Azure DevOps.

In this article, we will discuss the following points:

1- Requirements: Software that should be running to test this article.

2- Terraform scripts: all the files with its scripts.

3-Conclusion: The outcome of this article!

4-References: If you want to know more about this.

Requirements

Basic knowledge of Azure
You will need to have:

An Azure subscription
An Azure Devops project
A resource group already created
A storage account with one container
A Personal Access Token(PAT) generated from azure devops
A client ID and a client secret

3. Terraform should be installed and configured in the machine where the scripts will run

Terraform Scripts

The main objectives of these scripts are:

Creating one key vault per project, one for development and one for testing environment only.
Creating the service connections needed for Azure Devops.
Assigning all the permissions for the users who will administrate the key vaults.

In order to work in a efficient manner, we will have different terraform files.

variable.tfvars

A .tfvars file is a terraform file where you declare your variables. Then in your scripts files, you reference them writing var.$(variable_name).

So, in this file we are going to write all the variables to use:

pat= “xxxxxx” → personal access token created at azure devops

sub_id_dev= “xxxxxx” → your suscription id in azure

tenant_id= “xxxxxx” → your tenant id

client_id= “xxxxxx” → your client id

client_secret= “xxxxxx” → your client secret

sa_resource_group = “xxxxxx” → storage account resource group

storage_account= “xxxxxxx” → storage account name

container_name = “xxxxxx” → container name

key = “xxxxxx” → name of the file going to be saved at the container

org_service_url = “xxxxxx” → your organization URL in azure devops

00-main.tf

We configure our terraform required providers :

terraform {
  required_providers {
    azuredevops = {
      source = "microsoft/azuredevops"
      version = "0.1.0"
    }
    
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=2.46.0"
    }azuread = {
      source  = "hashicorp/azuread"
      version = "=2.11.0"
    }
  }
  backend "azurerm" {
  resource_group_name  = var.sa_resource_group
  storage_account_name = var.storage_account
  container_name       = var.container_name
  key                  = var.key
  subscription_id      = var.sub_id_dev
  tenant_id            = var.tenant_id
  }
}provider "azurerm" {
  features {}
}

01-providers.tf

Configuration of azure devops and azurerm providers:

provider "azuredevops" {
  org_service_url = var.org_service_url
  personal_access_token = var.pat
}provider "azurerm" {
  features {}
  subscription_id =  var.sub_id_dev
  tenant_id = var.tenant_id
  client_id       = var.client_id 
  client_secret   = var.client_secret
  alias = "non-prod"
}

02-variables.tf

Here we define the variables about each one of the projects. We can add the amount of projects we need:

kv_name = key vault name we will assign for that particular project
rg_dev = development resource group
rg_tst = testing resource group
proyecto_ado = azure devops project’s name
devops = user’s emails who will have access to modify the key vault.

This is an example, you have to replace all those variables according to your projects:

variable "p_data" {
    type = list(object({
      kv_name = string,
      rg_dev = string,
      rg_tst = string,
      proyecto_ado = string,
      devops = list(string)
  
  }))
  default = [
     {
      kv_name = "akvname" 
      rg_dev = "a-dev-rg"
      rg_tst = "a-tst-rg"
      proyecto_ado = "A_PROJECT"
      devops = ["devops1@gmail.com", "devops2@gmail.com"]
      },
     {
      kv_name = "bkvname" 
      rg_dev = "b-dev-rg"
      rg_tst = "b-tst-rg"
      proyecto_ado = "B_PROJECT"
      devops = ["devops11@gmail.com"]
     },
     {
      kv_name = "ckvname" 
      rg_dev = "c-dev-rg"
      rg_tst = "c-tst-rg"
      proyecto_ado = "C_PROJECT"
      devops = ["devops3@gmail.com", "devops4@gmail.com", "devops5@gmail.com"]
     }
  ]
}data "azurerm_subscription" "s_non_prod" {
  subscription_id = var.sub_id_dev
  provider = azurerm.non-prod
}

03-servicep.tf

To generate the integration between key vaults and azure devops projects, we will need to create a service principal for each key vault. This will allows us to create the service connections later.

data "azuread_client_config" "current" {}#Service principal
resource "azuread_application" "app-non-prod" {
  for_each = {for p in var.p_data:  p.kv_name => p} 
  display_name = "KV-${each.value.kv_name}"
  owners       = [data.azuread_client_config.current.object_id]
}resource "azuread_service_principal" "sp-non-prod" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  application_id               = azuread_application.app-non-prod[each.value.kv_name].application_id
  app_role_assignment_required = false
  owners                       = [data.azuread_client_config.current.object_id]feature_tags {
    enterprise = true
    gallery    = true
  }
}resource "azuread_service_principal_password" "sp-non-prod-passwd" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  service_principal_id = azuread_service_principal.sp-non-prod[each.value.kv_name].object_id
}

04-roleAssign.tf

We must assign the role “Contributor” to the previous service principals created in each one of the key vaults.

#Resource group from each environmentdata "azurerm_resource_group" "dev-rg" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  name = each.value.rg_dev
  provider = azurerm.non-prod
}data "azurerm_resource_group" "tst-rg" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  name = each.value.rg_tst
  provider = azurerm.non-prod
}# Assign Contributor permission to the service principalsresource "azurerm_role_assignment" "sp-dev-roleassignment" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  scope              = data.azurerm_resource_group.dev-rg[each.value.kv_name].id
  role_definition_name = "Contributor"
  principal_id       = azuread_service_principal.sp-non-prod[each.value.kv_name].object_id
  depends_on = [resource.azuread_service_principal.sp-non-prod]
  provider = azurerm.non-prod
}resource "azurerm_role_assignment" "sp-tst-roleassignment" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  scope              = data.azurerm_resource_group.tst-rg[each.value.kv_name].id
  role_definition_name = "Contributor"
  principal_id       = azuread_service_principal.sp-non-prod[each.value.kv_name].object_id
  depends_on = [resource.azuread_service_principal.sp-non-prod]
  provider = azurerm.non-prod
}

05-keyvaults.tf

In this file, we will create the key vaults per project. One per environment: dev and tst.

data "azurerm_resource_group" "rg_dev" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  name = each.value.rg_dev
  provider = azurerm.non-prod
}data "azurerm_resource_group" "rg_tst" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  name = each.value.rg_tst
  provider = azurerm.non-prod}resource "azurerm_key_vault" "kv-dev" {
  for_each = {for p in var.p_data:  p.kv_name => p}  
  name                        = "${each.value.kv_name}-dev"
  location                    = data.azurerm_resource_group.rg_dev[each.value.kv_name].location
  resource_group_name         = data.azurerm_resource_group.rg_dev[each.value.kv_name].name
  enabled_for_disk_encryption = true
  tenant_id                   = data.azurerm_subscription.s_non_prod.tenant_id
  soft_delete_retention_days  = 14
  purge_protection_enabled    = true
  provider = azurerm.non-prod 
  sku_name = "standard"
  depends_on = [azuread_service_principal.sp-non-prod]}resource "azurerm_key_vault" "kv-tst" {
  for_each = {for p in var.p_data:  p.kv_name => p}  
  name                        = "${each.value.kv_name}-tst"
  location                    = data.azurerm_resource_group.rg_tst[each.value.kv_name].location
  resource_group_name         = data.azurerm_resource_group.rg_tst[each.value.kv_name].name
  enabled_for_disk_encryption = true
  tenant_id                   = data.azurerm_subscription.s_non_prod.tenant_id
  soft_delete_retention_days  = 14
  purge_protection_enabled    = true
  provider = azurerm.non-prod
  sku_name = "standard"
  depends_on = [azuread_service_principal.sp-non-prod]}

06-sc.tf

Creation of the service connections in each Azure DevOps Project.

resource "azuredevops_serviceendpoint_azurerm" "sc-akv-non-prod" {
  for_each = {for p in var.p_data:  p.kv_name => p}
  project_id            = each.value.proyecto_ado
  service_endpoint_name     = upper("KV-${each.value.kv_name}-NON-PROD")
  description = "Managed by Terraform" 
  azurerm_spn_tenantid      = data.azurerm_subscription.s_non_prod.tenant_id
  azurerm_subscription_id   = data.azurerm_subscription.s_non_prod.subscription_id
  azurerm_subscription_name = data.azurerm_subscription.s_non_prod.display_name
  credentials {
    serviceprincipalid  = azuread_service_principal.sp-non-prod[each.value.kv_name].application_id
    serviceprincipalkey = azuread_service_principal_password.sp-non-prod-passwd[each.value.kv_name].value
  }
}

07-accesspolicy.tf

The access policy is the permission we give at each key vault to create, get, delete secrets and more.

In this file, we assign the access policy to all the users we defined in the 02-variables.tf with the list of permissions we require.

And we also assign the access policy to the service principals of the service connections with “get” and “list” permissions.

locals {flattened-pdata =  flatten([
    for pos, kv in var.p_data : [
      for pos2, devops in var.p_data[pos].devops : {
        kv_name = kv.kv_name
        devops_name = kv.devops[pos2]
    }]
  ])
}data "azuread_user" "user-ad" {
   for_each = {
    for devops in local.flattened-pdata : "${devops.kv_name}.${devops.devops_name}" => devops
    }user_principal_name = each.value.devops_name
}#Access policy assign to devops#DEV
resource "azurerm_key_vault_access_policy" "kv-pol-devops-dev" {for_each = {
    for devops in local.flattened-pdata : "${devops.kv_name}.${devops.devops_name}" => devops
    }key_vault_id = azurerm_key_vault.kv-dev[each.value.kv_name].id
      tenant_id = data.azurerm_subscription.s_non_prod.tenant_id
      object_id = data.azuread_user.user-ad["${each.value.kv_name}.${each.value.devops_name}"].object_idkey_permissions = [
      "get","list","update","create","import","delete","recover","backup","restore",
      ]secret_permissions = [
      "get","list","set","delete","recover","backup","restore",
      ]storage_permissions = [
      "get","list",
      ]  
      depends_on = [
        azurerm_key_vault.kv-dev
      ]
      provider = azurerm.non-prod
}#TST
resource "azurerm_key_vault_access_policy" "kv-pol-devops-tst" {for_each = {
    for devops in local.flattened-pdata : "${devops.kv_name}.${devops.devops_name}" => devops
    }key_vault_id = azurerm_key_vault.kv-tst[each.value.kv_name].id
      tenant_id = data.azurerm_subscription.s_non_prod.tenant_id
      object_id = data.azuread_user.user-ad["${each.value.kv_name}.${each.value.devops_name}"].object_idkey_permissions = [
      "get","list","update","create","import","delete","recover","backup","restore",
      ]secret_permissions = [
      "get","list","set","delete","recover","backup","restore",
      ]storage_permissions = [
      "get","list",
      ]
    
     depends_on = [
        azurerm_key_vault.kv-tst
      ]
      provider = azurerm.non-prod
}#Access policy assign to service connection's service principals.#DEV
resource "azurerm_key_vault_access_policy" "kv-pol-sp-dev" {
  for_each = {for p in var.p_data:  p.kv_name => p}key_vault_id = azurerm_key_vault.kv-dev[each.value.kv_name].id
  tenant_id = data.azurerm_subscription.s_non_prod.tenant_id
  object_id = azuread_service_principal.sp-non-prod[each.value.kv_name].object_idkey_permissions = [
      "get","list",
    ]secret_permissions = [
      "get","list",
    ]storage_permissions = [
      "get","list",
    ]depends_on = [
        azurerm_key_vault.kv-dev
    ]provider = azurerm.non-prod
}
#TSTresource "azurerm_key_vault_access_policy" "kv-pol-sp-tst" {
  for_each = {for p in var.p_data:  p.kv_name => p}key_vault_id = azurerm_key_vault.kv-tst[each.value.kv_name].id
  tenant_id = data.azurerm_subscription.s_non_prod.tenant_id
  object_id = azuread_service_principal.sp-non-prod[each.value.kv_name].object_id
  
   key_permissions = [
      "get","list",
    ]secret_permissions = [
      "get","list",
    ]storage_permissions = [
      "get","list",
    ]
  
   depends_on = [
        azurerm_key_vault.kv-tst
    ]
   provider = azurerm.non-prod
}

Script execution

To execute these scripts you have to write in the command line the following steps:

terraform init → it initializes the terraform.
terraform plan -var-file=variables.tfvars → it generates a plan in the command line about the changes it would do, using the file with your variables “variables.tfvars”.
terraform apply -var-file=variables.tfvars → it will apply your changes if you confirm to “yes”. See the example:

Conclusions

Terraform is a powerful tool to maintain a big amount of services in our infraestructure. This was only one example about how we can use Terraform to mantain key vaults but we can apply this solution in any other services. In this way, we are efficient, save a lot of time, and have more control over our key vaults.

References

Thanks to Pablo Rubini for the help in the creation of this tasks.