Goodbye template files! Using CDKs to migrate from Infrastructure as Code to Infrastructure as Software

Published in

DVT Software Engineering

6 min readApr 21, 2022

Introduction and a brief history of cloud infrastructure

The way we provision this new cloud infrastructure is constantly evolving, all with the same aim of faster, more reliable delivery. Infrastructure as Code (IaC) is one of the key paradigms to ensuring this speed, quality, and reliability.

Instead of relying on manual clicks on a “portal” or “console” — which leads to the very real risks of poor auditability, poor consistency, errors, and key-man dependencies — why not use templates instead?

Azure ARM Template overview. Source: https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/overview

Templates offer you a way to version your infrastructure; they’re also self-documenting, allow for automation (which saves time and reduces error), and allow for some finer controls — such as pull requests in an IaC Git repository, for example. Azure offers Azure Resource Manager, and AWS offers CloudFormation — both are configuration languages that can be used for writing IaC template files.

These configuration languages also, unfortunately, tend to be a bit of a pain to use at the enterprise level: their JSON/YAML based syntax can make them verbose and somewhat challenging to work with from the standpoint of legibility, modularity, validation, and testing.

AWS CloudFormation snippet: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-cloudformation.html

Fortunately, the IaC world has progressed, and one of the latest and most exciting innovations to emerge is the Cloud Development Kit.

Cloud Development Kits (CDKs) allow us to utilise the same general-purpose programming languages we use to build our applications — C#, TypeScript, Go, Python, Java, and others — for our IaC. The templates and configuration languages behind them are no longer of concern to your cloud engineers. Instead, the CDK enables the infrastructure source code to be transpiled into these configuration languages without requiring an engineer to touch a template file ever again.

Using these CDKs is as easy as installing a library/package.

There are a few notable names offering mature CDK products (as well as others catching up):

AWS CDK

The latest AWS CDK is AWS CDK v2, and currently supported languages include TypeScript, JavaScript, Python, Java, and C#, (with Go support currently only available in the developer preview edition)

Creating an S3 bucket in C# using the CDK now becomes as simple as the below:

AWS CDK C# code snippet: https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html

Pulumi

The previously mentioned CDK tool is vendor-specific. It’s good news if your current IaC stack is on AWS, but what about the other major public cloud providers? i.e., Microsoft Azure and Google Cloud Platform?

Pulumi is one of the possible options, and it provides CDK tooling for the above cloud providers (including AWS), as well as Kubernetes

It’s open-source and has different tiers for the product (Individual, Team, and Enterprise).

Language runtimes currently supported include Node.js (JavaScript, TypeScript), Python, Go, and .NET Core (C#, F#, VB)

Creating an Azure Storage Account using Pulumi in C# becomes as simple as below:

Pulumi C# snippet: https://www.pulumi.com/docs/intro/languages/dotnet/

Terraform CDK

HashiCorp Terraform has built a strong name for themselves in IaC, and they haven’t allowed themselves to be left behind when it comes to CDK offerings

This is another option for CDK across multiple cloud providers, and support is included for AWS, Azure, and Google Cloud.

Current language support includes TypeScript, Python, Java, C#, and Go

Creating an AWS EC2 instance using Terraform CDK can be as simple as below:

CDK for Terraform Python snippet: https://learn.hashicorp.com/tutorials/terraform/cdktf-build-python?in=terraform/cdktf

Unfortunately, CDK for Terraform is currently in beta; however, it’s worth mentioning and keeping an eye on for those already making use of the popular Terraform HCL.

Sourced from Terraform’s official documentation: “This tool can be used with Terraform Cloud and Terraform Enterprise, but is not eligible for commercial support, and is not officially recommended for production use cases.”

Benefits of using a general-purpose programming language to provision cloud infrastructure

If you’re from a developer background, ridding yourself of bulky YAML/JSON files might be enough to jump right into the CDK world. Still, the benefits of this toolkit are a bit more profound than mere developer preferences.

What CDKs essentially provide to us is the entire ecosystem of the programming language for infrastructure provisioning. Out of the box, you are now getting core programming language features that are either unavailable or not as refined with previous IaC configuration languages:

variables
control flow statements (for loops, while loops etc.)
methods and functions
data structures
typing
compilation (for compiled languages)
exception handling
stack traces
polymorphism
functional and/or object-oriented paradigms, etc

In addition, you are now able to apply software development principles and practices to infrastructure provisioning:

Unit testing and mocking

This can enable you to catch errors before you even commit code, never mind deploying it

Design patterns and principles

You’re better empowered to write cleaner code, and you can apply principles such as SOLID to your infrastructure codebase

CDKs: Even better for DevOps!

CDKs provide the opportunity to eliminate the disjointedness of application and infrastructure source code. Your team has identified an application technology stack, and with an installation of a package or two, that same stack can be extended towards infrastructure.

There’s less cognitive overload for your engineering team. The IaC domain-specific languages of yore use their unique concepts, syntax, and best practices. These require time to learn, and they take time to implement and validate. In DevOps, we’re trying to be faster and better.

Using the same tools for applications and infrastructure removes hurdles to a shared understanding of the system. Having been on both the dev and ops side myself, I understand why developers become intimidated and steer clear when encountering a 3000-line bulky YAML file describing infrastructure.

CDKs allow dev and cloud-ops to understand one another’s work better and review each other’s work. They carry the potential of further eliminating the distance between the two worlds.

Even on a day-to-day working basis — that IDE you love for your application code is now the same one you can use for your infrastructure:

- Sure, syntax highlighting, IntelliSense, linting are possible with JSON and YAML, but it’s just so much easier and better with a programming language.

These benefits apply to your CI/CD pipelines as well. The same tests you write for your application can now be written and executed for your infrastructure. The additional effort to ascertain quality gates for YAML and JSON files is no longer required.

CDKs carry with them the potential of providing: