PowerShell Azure API functions in Terraform
tl;dr How to call the Azure API directly via PowerShell from within Terraform
For the most part Terraform does what I need, but there have been times where it just can’t manage some things well. For this post I am pointing at you, “azurerm_security_center_workspace”, yes, you. For the last year you’ve been annoying to deal with, like executing resource deployment for an hour before throwing your hands up in the air and giving up 😤.
This issue had been raised over a year ago, https://github.com/terraform-providers/terraform-provider-azurerm/issues/5475, and I think longer than that in other posts. This causes massive delays, retries in pipeline runs and is just downright annoying.
Finally, after having to write some Terratest code for it and watching it fail often, I gave up and started to look elsewhere to deploy the resource. I tried doing it via an ARM template Terraform resource, but it too failed. It failed within minutes, which is far better than the 30–60min it was doing via Terraform, but it still failed.
The Solution!
I must thank a colleague who pointed me in the right direction of writing a PowerShell API function that does the same thing. At first I thought “Surely, an ARM template would work better? They’re calling the same API!”, but it seems that is not the case, and directly calling the Azure API is a whole lot faster… and more importantly, it works! 😀
For this post I am referencing the Azure REST API documentation found here: https://docs.microsoft.com/en-us/rest/api/azure/
I’ll cut to the chase and flip up some code... The first part is the null_resource for Terraform that calls the PowerShell functions, importing the function into memory and then calling each function by passing in the required parameters.
URL Encoding
As you can see you need to authenticate with Azure first and then pass the bearer token into the API request for authorisation. But the most important thing I want to bring to your attention is the .NET URL encoding on line 80; [System.Web.HTTPUtility] . As with the case of passwords, there will be situations where some will have special characters, and if that is not properly converted to the “content type” of x-www-form-urlencoded formatting of the API call, it will fail.
The best way to understand what I mean is by looking at line 81, where the scope contains https%3A%2F%2Fmanagement.azure.com%2F.default , which in non-url-encoded speak is actually https://management.azure.com/.default . Without the conversion, the “content type” of the API call would just fail, and that would be the same case for passwords with special characters, too. After that the rest is pretty self-explanatory; deploy the thing, or destroy the thing, and the thing can by anything you want that is supported by Azure REST API (as linked above).
Destruction!
If you were one of keen eye you would notice an extra parameter when = destroy on the destroy null_resource . This is the cool part, where you can run the DELETE API function to remove the resource you deployed upon using Terraform Destroy. As far as I know, you must pass some triggers into the provisioner to use it, even if you’re not really “triggering” anything. It’s the requirement of the when = destroy function that must call self.xxx resources, which in our case is just the same value parameters passed down. Of course one downside to all of this is that your credentials are written to state, so please ensure that is secure from prying eyes.
Thoughts…
I hope this gives you ideas on how you can run your API calls from within Terraform, Azure or not. We have used this same ability to connect deployment agents to Azure DevOps Agent Pools via it’s “managed service” to scale up/down your Virtual Machine ScaleSet, as this feature is not yet available in Terraform (or even Azure cmdlets!).
