How Azure Bicep is Different
How emerging technologies solve classic problems in Infrastructure-as-Code.
What is Azure Bicep?
Azure Bicep is a new domain-specific language (DSL) to manage Azure infrastructure-as-code. With a more flexible and concise syntax than JSON, it reduces the size of code and improves modularization. Bicep also supports every workflow where ARM (Azure Resource Manager) templates are used, which can ease the transition to Bicep for teams already using ARM templates.
The idea behind Bicep marks a new approach to address a known challenge in infrastructure as code. To understand what sets Azure Bicep apart, let’s review what makes infrastructure-as-code challenging.
The challenge with infrastructure-as-code
Traditional infrastructure-as-code (IaC) schemes include a deployment engine and a declarative template. Users write the template as a spec sheet. The deployment engine (provided by the platform) then translates the template into API requests issued to the underlying hypervisors. Many first generation IaC tools are variations of this scheme, such as CloudFormation and ARM, which are widely adopted in infrastructure management today.
In their declarative template, users only specify the end state of desired resources. This allows the deployment engine to perform idempotent operations, which reduces complexity for users. The declarative template is written in configuration code, either YAML or JSON. Unfortunately, these markup-based languages create a pain point when describing specifications.
Markup languages are static by nature. In the real world, we often want to dynamically compute attribute values in a self-defined function to avoid hardcoding. Markup languages fall short in this respect. Instead, users rely on the cloud providers to implement frequently used functions (e.g. string concatenation) on the deployment engine side, and call the function from within their templates. Azure refers to these functions as template functions and AWS refers to them as intrinsic functions. Each provider has a short list of available functions for their users to choose from, and creation of new functions always lags behind users’ needs.
Another challenge with markup code is the sheer size. Declarative statements are inherently wordy, with every sub-attribute requiring a line of code that can’t be reused. For example, declaring 4 TCP ports for an ingress security group in CloudFormation takes a minimum of 16 lines (4x4) in YAML. To cope with this issue, it is advised to divide up a large template into several smaller ones (using nested templates), which brings the new problem of referencing resources between smaller stacks (cross-stack referencing). As a result, we have to add even more lines of code to each smaller stack because markup languages lack proper modularization. For example, with a nested stack, authors have to add least four lines in each reference in CloudFormation and ARM. As more resources are incorporated, and more dependencies are introduced, the templates quickly sprawl beyond what is human-readable.
Markup languages are too inflexible and verbose to express the resource interaction during infrastructure provisioning today. As a result, market players have devised the following approaches.
Approach 1. Design a new language
HashiCorp’s Terraform is a forerunner in attempting to address those challenges. It launched early in 2014 and has gained a considerable customer base. In June 2021, it released 1.0 for General Availability. To start with Terraform, users need to learn Terraform’s own DSL, the HCL (HashiCorp Configuration Language). In return, they can modularize resources through simple file-splitting without the overhead of nested stacks. Cross-stack resource referencing is done in one line. The HCL syntax is also more concise than YAML or JSON, with support of if-statements and for-loops. However, Terraform introduces new challenges of its own.
Terraform connects to different platforms (e.g. Azure, AWS) through providers. The providers are operated as open-source projects in collaboration with each platform. This platform-specific provider introduces an intermediary layer between the HCL code and the APIs of the platform. The maintenance of this layer itself poses a risk. This layer may introduce bugs. It may cause delays in functional delivery. When a new resource type, or a new attribute becomes available from the platform API, it takes time for the Terraform provider to catch up.
The other overhead Terraform brings in is the need for state management. A state file keeps information about current infrastructure (including sensitive data). In a team with multiple authors, the state file must be managed securely and remotely, with a locking mechanism to avoid race conditions. For these challenges, HashiCorp introduced its own deployment platform, Terraform Cloud (as a commercial SaaS) or Terraform Enterprise(the self-hosted alternative), as an option for teams that can afford the additional cost or operation overhead.
Approach 2. Integrate with general-purpose language
AWS created Cloud Development Kit (CDK) to approach the challenges presented by existing IaC solutions. Instead of modules, CDK introduces the concept of constructs: reusable collections of cloud objects. Instead of a new DSL, CDK developers can choose a popular general-purpose language such as Java, Python or TypeScript. The integration with widely accepted languages brings lots of options that are not available with markup languages or HCL. For example, you can follow an object-oriented design pattern (OOP), import third-party libraries in Python, or incorporate infrastructure into your existing framework for unit testing.
Pulumi is another technology that falls into this category. It supports many of the same languages as CDK but works with a variety of platforms (AWS, GCP, Azure, Kubernetes, etc).
Because infrastructure capability is embedded into the general-purpose code, IaC offerings in this category fit perfectly with application development projects where infrastructure is a built-in part of the release (also referred to as Infrastructure as Software by its advocates). For this sake, both Pulumi and AWS CDK seek to support as many general-purpose languages as possible to expand their use cases.
Despite its flexibility, this approach may not be an optimal experience for operation-oriented teams. The use of object-oriented programming forms another layer of abstraction. The code may come off as cleaner to developers, but less straightforward for infrastructure administrators when they try to picture the stack in their mind. This is because the code has to be organized in a programming pattern such as OOP. For the same reason, starting an infrastructure project in CDK or Pulumi takes longer, and more programming knowledge is required. This example gives an idea of the overhead before spinning up a simple RDS service with CDK.
Approach 3. Enhance existing language with advanced features
This is where Azure Bicep fits in. The Bicep DSL is an enhancement of JSON configuration language. It introduces some flow-control elements such as loops, ternary operators and one-line resource references, yet it stops short of classes or inheritance. The enhanced syntax greatly improves functionality and reduces code size, at the cost of mildly increased complexity. As an Azure-native technology, Bicep also has two advantages over approach one: day-zero support and no maintenance of state files.
Compared to approach two, Bicep wins through low cost of adoption. Simplicity makes Bicep DSL incredibly easy to learn. All the features are covered in a seven module tutorial. People with an operation background can pick it up without a serious training on programming. Additionally, it is completely compatible with its predecessor, ARM templates. A team already leveraging ARM templates can introduce Bicep files without changing the Azure CLI commands. This will ultimately reduce the transition effort for an ARM-driven operation.
To to illustrate Bicep’s simplicity, here is an example to provision the core resources for an enterprise data warehouse in Azure.
On the downside, Bicep DSL’s flow-control syntax is primitive when compared to “real” programming languages. This limits its use to operations only. With Azure Bicep alone, you are not able to develop an application with the capability of provisioning infrastructure.
The Era of Choice
Today, the discussion is not about whether you should use infrastructure-as-code. Rather, it’s about which approach you should take to suit the needs of your specific project and organization. New IaC technologies have diverged to fulfill different use cases. If your team is developing an application with associated infrastructure, CDK will enrich the “Dev” experience with support of popular programming languages, and Pulumi provides the same benefits with cross-platform support. On the other hand, if your operation team is currently using ARM templates, Azure Bicep will help refine the “Ops” processes.
