Manage your (Azure) infrastructure with Terraform | Part 1
This blog starts a series on the topic of infrastructure as code (IaC) with Terraform from HashiCorp. It includes an introduction to the general IaC aspects and which IaC approaches are natively available for Microsoft Azure, which will also be the focus of the further content on Terraform.
Companies are still struggling, partly with the effort of also having their infrastructure code-based. Why this is totally worthy hopefully gets clearer afterwards.
What is infrastructure as code, and what are the benefits of the approach?
Resources in the Azure Cloud or another hyperscaler can be created, configured, and deleted in a variety of ways. An easy and convenient way is, for example, a GUI such as the Azure portal. It is clear and often supports deployment or changes with wizards. With Azure, there is also the option of working with the native command-line tool Azure CLI or using modules from PowerShell. Furthermore, various SDKs are available, e.g., for C#, Java, or Python. In the end, everything ends up with the Azure Resource Manager, the central API.
The use of GUI in important productive environments is only recommended to a limited extent, as this method has some disadvantages. Creating resources may be so simple, but later questions arise: how was the resource created, and who did it? Is it still the original configuration, or has it been changed, and is it valid and in accordance with set standards? How long does it take to go through the wizard's portal? What if you want to repeat a deployment? Resources could be accidentally deleted or misconfigured. The Azure logs can help with the search for clues, and policies can be used to secure security and governance standards. However, this only addresses some of the problems. In addition, resources should be deployed in the same way to different stages, such as development, testing, and production, as is common in enterprise IT. This is very time-consuming and error-prone and can lead to stand-alone, isolated solutions, or "snowflakes,” if the environments are no longer consistent.
To maintain a stable cloud environment, there is no way around IaC. Instead of creating the resources in the portal with many clicks and not knowing how this happened later, code is written, stored in a repository, and usually provisioned from there.
The advantages are obvious and will be known to every developer: It is traceable what was done and when; there is versioning. Teamwork is possible, and with the appropriate parameterization, the environment can be easily rolled out to different stages. Deployments can be automated and easily repeated. To avoid the disadvantages mentioned, there is, therefore, often also a requirement in projects to deploy cloud resources only via IaC. In this case, the portal should usually only be available for troubleshooting for certain users. With DevOps and pull request strategies, standards for configuration can also be supported. A resource order within the company, let’s take a virtual machine, is no longer a tedious process but can be automated with a template-based process. The development of the code takes time, but troubleshooting and manual processes usually take even more. IaC provides security and reproducibility, e.g., with Terraform.[2]
IaC Methods and Approaches
To round off the basic IaC description, a few terms are explained that are often used in this context:
- Push vs. Pull
- Idempotence
- Declarative vs. imperative
The first point describes the way in which the configuration of the infrastructure is transferred: Active by commands (push)—here, Terraform is to be classified — or passively (pull), where the configuration is stored at a central location and then picked up by a kind of agent. Idempotence means that a consistent state of the target system is maintained and corresponds to the configuration in the code, no matter how often a deployment is executed. If nothing has changed in the code, nothing changes in the deployment target. In addition, Terraform takes a declarative approach. This means that instead of saying step by step and command by command exactly what is to be done (imperative), only the desired end state is described. The difference between the actual and target states is then determined by tooling, and the corresponding changes are made. The exact sequence of the commands then no longer bothers the user in detail[3].
Azure IaC — Terraform, Bicep, ARM
The focus of this series of posts, Terraform, is just one option for using IaC. While Terraform takes a cross-cloud provider approach, there are also native options with hyperscalers. In the area of the Azure Cloud, these are the Azure Resource Manager (ARM) templates in their original form. These are JSON-based files that define the resources and parameters to be created. These files can then be parsed, and the content processed by the Resource Manager. Such a template can be exported from each created resource via the Azure portal. The number of lines of code is significantly higher compared to other options and defining dependencies correctly can be complicated. Although they are no longer considered up-to-date for a complex IaC approach, they are by no means indispensable. Under the hood, they are often still used, for example, in Azure policies or in services such as the Azure Data Factory, where configuration changes are processed by generated ARM templates. Azure’s own, more modern approach is Bicep. This is a wrapper that significantly simplifies the native Azure development of IaC scripts. The language offers many helpful functions and can make dependencies between resources much easier to write or handle independently. In the end, ARM templates are generated for deployment, which are then used to communicate with the API. [4] In practice, these three options are often combined with PowerShell or an Azure CLI task throughout the context of a deployment, such as an Azure DevOps pipeline. Not every desired setting is possible or practically feasible with Terraform or Bicep.
Terraform — Managing Different Infrastructures
Terraform comes from the company HashiCorp and offers the user the possibility to implement IaC for many providers. Many common providers are already available as plugins. In addition to the Azure Cloud, Amazon AWS, or Google Cloud, these include Docker and Kubernetes, among others. In total, there are now over 1,000 of these plugins. If the required provider is not on this long list, it can be extended by your own plugin. Terraform works with the HashiCorp Configuration Language (HCL), which defines the basic structure and can be dynamized with expressions and template functions. The exact specification can be found open-source on GitHub[5], as well as the Azure provider[6] for the resources of the Microsoft Cloud. Terraform is currently in version 1.4.6 and has become very stable.
Terraform Basics
Configuration — Structure of the HCL code
Terraform is referred to as “configuration”, but it is not only parameters or the like; it refers to the entire IaC code, including resources to be created or data retrieval. Usually, there are a couple of blocks in the files. A block follows the structure Type — Resource Type — Reference Name. Within a block, settings are defined via key-value pairs or special embedded blocks for object types.
As types for the value definitions, Terraform recognizes strings, numbers, booleans, arrays, and objects:
Terraform CLI
Terraform works with a bunch of files with specific endings. When you double-click on a Terraform file, nothing happens, or an editor opens. To make the desired changes or queries, the Terraform CLI is used. With any command-line tool, the corresponding instructions for processing by Terraform can be sent via the “Terraform” command as an interface. Very helpful when using the CLI is the “-help” command, which works at different levels, e.g., directly at the root level or specific to a sub-command like terraform init -help.
Relevant for the execution of the Terraform configuration with the console is the current path from which the commands are issued, as well as everything that is underneath. All folders and files are automatically included.[7]
Terraform Documentation
Detailed documentation on how to address the available provider modules — not only for the cloud — can be found under registry.terraform.io, Hashicorps public repository. According to a uniform scheme, as in the example of a storage account [8], references and examples can be used to find out which attributes can or must be specified with the appropriate syntax. In addition, the documentation describes which data queries are possible for the resources and how an import for the important state file works.
The central documentation for how to deal with specific language functions or tutorials can be found at: https://developer.hashicorp.com/terraform.
Summary
This blog gave an overview of Infrastructure as Code as a general topic, what’s available in the area of Microsoft Azure, and why it’s so important. Coding and repositories are not exclusive to C# object-oriented programming or modern web applications. Companies should, if not already doing so, start this way for their infrastructure as well. Otherwise, IT risks a lot of problems, not only from hackers outside.
Furthermore, we shed some light on how Hashicorp’s Terraform approach works, which covers a big bunch of modules for different providers. Native options or Terraform: at least one option for IaC is strongly recommended.
In the next post of the series, we will take a look at the initial terraform setup and explain in detail the steps of an example deployment.
[1] https://learn.microsoft.com/de-de/azure/azure-resource-manager/management/overview
[2] https://www.atlassian.com/de/microservices/cloud-computing/infrastructure-as-code
[3] https://www.redhat.com/de/topics/automation/what-is-infrastructure-as-code-iac
[4] https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview?tabs=bicep
[5] https://github.com/hashicorp/hcl/blob/main/hclsyntax/spec.md
[6] https://github.com/hashicorp/terraform-provider-azurerm
[7] https://developer.hashicorp.com/terraform/cli/run
[8] https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/storage_account#example-usage
Originally published at https://thomashafermalz.net on May 31, 2023.
