Configure Azure Virtual Network peerings with Terraform
Azure Virtual Networks (or short VNets) are the fundamental building block for all network and network-related configurations in Microsoft Azure. They provide a logical boundary allowing you to secure network communication between the attached Azure resources. By default these resources can talk to each other and out to the Internet, but what if you want to communicate with resources in another Virtual Network? Virtual network peering enables you to connect two or more VNets within the same Azure region or across different Azure regions (Global virtual network peering).
Virtual networks as well as virtual network peerings can of course be defined, deployed and configured via Terraform and other Infrastructure-as-Code toolkits like any other Azure resource. Following the guidance of frameworks like Microsoft’s Cloud Adoption Framework (CAF) and Enterprise-Scale (ESLZ) or even to implement a simple Hub-and-Spoke architecture are these resources often times spread across multiple Azure subscriptions and in some cases even across Azure subscriptions associated with different Azure Active Directory (Azure AD) tenants.
Terraform supports these kinds of deployments using multiple Terraform Provider definitions but there are some caveats and pitfalls you should know. In the subsequent sections of this blog post I’ll walk you through the most complex scenarios.
Important! All code samples are kept intentionally simple, in real deployments these components should be implemented as modules to make them re-usable.
Single Tenant, Multiple subscriptions
The first scenario is pretty straight forward, we have two Azure subscriptions attached to a single Azure AD Tenant. Following the princple of least privilege, we have two different Service Principals, each of them has access to one specific subscription only.
The diagram below shows us the high-level architecture of our scenario. We have a single Azure AD Tenant called “Tenant A”, two Azure subscriptions “Site A” and “Site B” and two Service Principals, each of them with access to only one of the two subscriptions.
Access in above’s example means Contributor permissions on subscription-level to create Azure resources within the subscription. And Network Contributor in the remote subscription to authorize the creation of the VNet peering. You’ll find a more detailed description further down below.
In Terraform the AzureRM provider for Azure Resource Manager (ARM), and probably the ones for other cloud providers like AWS and GCP as well, is typically attached to a single subscription. For deployments to multiple subscriptions we therefore need multiple provider definitions. This is the case as soon as you’re addressing more than one Azure subscription OR in case you’re deploying resources with different Service Principals within the same Azure subscription.
The first “azurerm” provider definition is our “default” provider, this definition will be used when in a resource definition no provider is specified.
Note! I had some trouble assigning the “default” provider definition its own “alias”. Therefore i left it blank. No provider (in all subsequent resources) means therefore “default”.
provider "azurerm" {
features {}
}The second “azurerm” provider definition is representing the “siteb” subscription:
provider "azurerm" {
features {}
subscription_id = "secondary-subscription-guid"
client_id = "client-id"
client_secret = "client-secret" alias = "siteb"
}
The main differences between these two provider definitions are obvious, for the first one (the default one) we specify nearly nothing and we are relying on the Azure credentials handed over via the environment variables:
- ARM_SUBSCRIPTION
- ARM_CLIENT_ID
- ARM_CLIENT_SECRET
- ARM_TENANT_ID
The “siteb” provider definition points to a different Azure subscription by specifying subscription_id and uses a different Service Principal by specifying the client_id and client_secret to access our “siteb” subscription. The Azure AD tenant remains the same (by not specifying it) and is set via the ARM_TENANT_ID environment variable for both (all) provider definitions.
Important! Secrets should of course not be stored in Terraform code. You can load them from a secret store (like for example GitHub secrets) and inject them via environment variables into the build environment.
Now that we have specified the “siteb” azurerm provider definition, we have to make sure that all resources that are going to be deployed into “siteb” are using our “siteb” provider definition instead of the “default” one.
Here is an example of how this looks like for an Azure resource group and an Azure virtual network:
# Azure resource group for site b
resource "azurerm_resource_group" "site_b" {
name = "peering-siteb-rg"
location = "westeurope" provider = azurerm.siteb
}# Azure virtual network deployment for site b
resource "azurerm_virtual_network" "vnet_b" {
name = "peering-vnet-b"
resource_group_name = azurerm_resource_group.site_b.name
location = azurerm_resource_group.site_b.location
address_space = ["10.20.0.0/16"] provider = azurerm.siteb
}
The peering itself is then configured in a pretty similar way. We have peer_a2b in our “sitea” subscription without a provider definition (using the default one) and peer_b2a in our “siteb” subscription using the “siteb” provider definition:
# Azure Virtual Network peering between Virtual Network A and B
resource "azurerm_virtual_network_peering" "peer_a2b" {
name = "peer-vnet-a-with-b"
resource_group_name = azurerm_resource_group.site_a.name
virtual_network_name = azurerm_virtual_network.vnet_a.name
remote_virtual_network_id = azurerm_virtual_network.vnet_b.id
allow_virtual_network_access = true
}# Azure Virtual Network peering between Virtual Network B and A
resource "azurerm_virtual_network_peering" "peer_b2a" {
name = "peer-vnet-b-with-a"
resource_group_name = azurerm_resource_group.site_b.name
virtual_network_name = azurerm_virtual_network.vnet_b.name
remote_virtual_network_id = azurerm_virtual_network.vnet_a.id
allow_virtual_network_access = true provider = azurerm.siteb depends_on = [azurerm_virtual_network_peering.peer_a2b]
}
This works like a charm and will spin up two Azure resource groups in two different subscriptions, two virtual networks and a peering in between. The subscriptions are accessed via their individual Service Principal.
Two Tenants, Two Subscriptions
Our second scenario is a bit more complex, the configuration is still pretty much the same. This time we have two Azure subscriptions associated with two different Azure AD tenants and two different Service Principals with Multi-tenancy enabled.
The setup here is nearly the same as it was in our first scenario. The difference is that our previous implementation and provider definitions will not work when our Azure subscriptions are associated with two different Azure AD tenants.
Even though our azurerm provider definitions are pointing to the two different subscriptions, using two different Service Principals, and all permissions are set, the deployment of our Virtual network peerings will fail.
Here is the error message you can expect when TF tries to create the peering:
│ Error: network.VirtualNetworkPeeringsClient#CreateOrUpdate: Failure sending request: StatusCode=0 — Original Error: Code=”LinkedAuthorizationFailed” Message=”The client has permission to perform action ‘Microsoft.Network/virtualNetworks/peer/action’ on scope ‘/subscriptions/***/resourceGroups/tfpeer-sitea-rg/providers/Microsoft.Network/virtualNetworks/tfpeer-vnet-a/virtualNetworkPeerings/peer-vnet-a-with-b’, however the current tenant ‘***’ is not authorized to access linked subscription ‘***’.”What you will see in your subscriptions is, that TF was able to deploy the resource groups and virtual networks, but failed to deploy the peerings. The reason for that is that is that during the creation of a virtual network peering across different tenants, each AzureRM provider needs to be able to authenticate against the other tenant. And this is where “auxiliary_tenant_ids” (if you want to learn more about cross-tenant authentication, check out the Azure RM documentation) comes to play.
“auxiliary_tenant_ids” is an additional, barely documented, argument for the AzureRM provider in Terraform. To enable cross-tenant peering deployments, each provider needs the other, remote tenant in its list of “auxiliary_tenant_ids”.
provider "azurerm" { # Site A
features {} auxiliary_tenant_ids = ["tenant-b-guid"]
}provider "azurerm" { # Site B
features {} subscription_id = "secondary-subscription-guid"
client_id = "client-id"
client_secret = "client-secret"
tenant_id = "tenant-b-guid" auxiliary_tenant_ids = ["tenant-a-guid"] alias = "siteb"
}
This will make sure that the AzureRM provider authenticates against its own Tenant and will also get the authentication token for other tenants specified via the “auxiliary_tenant_ids” argument. The Service Principal we are using must have been invited as a guest to the other tenants.
And this brings us to the third and last part of our blog post, the creation and preparation of our Service Principals used for cross-tenant deployments.
Create and prepare Service Principals for Multi-tenancy
The process of creating and “Authenticating using a Service Principal with a Client Secret” (with a Managed Service Identity, Service Principal and a Client Certificate or Azure CLI) is documented well in the AzureRM documentation on terraform.io. Specific to our scenario are:
- Enable Multitenant access
- Consent
- Permissions
Enable Multitenant access
Azure App Registrations (Service Principals) are by default configured for “Single tenant” use. To enable cross-tenant scenarios like ours we need to change that.
- Azure AD > App registrations > Authentication > Supported account types
This needs to be done for both Service Principals in “Tenant A” and “Tenant B”. Next task is to create each of the Service Principals in the other tenant. This is done via consent.
Consent
To create our Service Principals in the foreign Tenant we have to call the consent URL for each of our Service Principals.
To create the SP from Tenant B in Tenant A:
https://login.microsoftonline.com/<tenant-a-id>/adminconsent?client_id=<siteb-client-id>To create the SP from Tenant A in Tenant B:
https://login.microsoftonline.com/<tenant-b-id>/adminconsent?client_id=<sitea-client-id>This operation needs administrative permissions in each of the Azure AD tenants. After you have completed this task should both Service Principals show up in the other Tenant and you can assign permissions and roles to them.
Permissions
As mentioned above do we need some specific permissions for both of our Service Principals to make our cross-tenant deployment work.
Tenant A / Service Principal
- Contributor on subscription-level in Site A to deploy Azure resources
- Network contributor on subscription-level in Site B to authenticate against the other tenant and to create virtual network peerings
Tenant B / Service Principal
- Contributor on subscription-level in Site B to deploy Azure resources
- Network contributor on subscription-level in Site A to create virtual network peerings
For more details which permissons are needed to configure virtal network peerings, check out the Permissions section in Create, change, or delete an Azure virtual network peering on Microsoft Docs.
I hope this was helpful. Thank you very much for your attention!
