Terraform Vs AWS CDK
Infrastructure as Code (IaC) has revolutionised the way that we deploy infrastructure by making it easy to automate processes that previously would have been manual, time consuming, and error prone.
There are many different IaC tools that try to solve the problem in slightly different ways and each come with their own strengths and weaknesses.
In this blog post I shall cover the two main tools that we use at Nationwide Building Society in the Future Platform team, and some of the lessons that we have learnt using them.
Terraform and CDK basics
Syntax
Terraform uses a declarative syntax called HCL (HashiCorp Configuration Language) for defining infrastructure. The main benefit of this syntax is that it is easier for humans to work with than JSON.
Although HCL is fairly simple to learn it can quickly get quite complicated when trying to do seemingly basic things such as looping over lists of data. It is an additional language and way of thinking that a developer will need to get familiar with to become productive.
With AWS CDK you can use a SDK for the language that you are most familiar with such as TypeScript, Python, Java, C#/.Net, and (in developer preview) Go.
The syntax of AWS CDK is imperative which makes it feel more like you are describing how to provision the infrastructure, however it ultimately gets converted into CloudFormation which is declarative.
The main benefit of using AWS CDK is that developers do not need to learn a new language and can benefit from all the features of their preferred programming language.
State
Terraform needs to keep a record of the real world infrastructure that has been provisioned alongside the desired infrastructure that has been specified in code. This is achieved by writing data to a file called terraform.tfstate.
The state file also provides additional benefits such as storing data about the order of dependencies in order to create and destroy infrastructure in the correct order when desired.
Terraform state can be stored remotely such as in S3. This is especially useful when working in a team as it ensures that each team member is working with the latest version of the state.
In contrast, AWS CDK does not use state and instead creates CloudFormation stacks. The complexity of managing the creation and destruction of the infrastructure is therefore offloaded to CloudFormation.
CLI commands
Both Terraform and AWS CDK provide CLI commands for various operations such as validating the specified infrastructure, checking what changes will be made, applying the changes, and destroying everything.
Terraform provides additional commands for manipulating the state, which CDK does not provide as it does not have state.
Differences between Terraform and AWS CDK
Platform support
Terraform is cloud agnostic and extendable. This is achieved using “providers” which are an abstraction on top of the underlying API. This blog post mainly concerns use of the AWS provider for Terraform. Although you can write Terraform for multiple cloud providers, the HCL code that you write is coupled to a specific cloud provider and not cross-cloud compatible.
AWS CDK is locked to AWS and cannot be used for any other cloud. Hashicorp have released their own CDK which allows you to call any Terraform provider from CDK code, including for other cloud providers, however this is not possible from AWS CDK. There is also CDK8s for working with Kubernetes however this is a separate tool.
Constructs and levels
CDK has a concept of “constructs” which are the building blocks of CDK apps. The constructs come in three different “levels” which are layers on top of CloudFormation.
Level 1 constructs are Cfn (short for CloudFormation) resources and map directly to the resources available in CloudFormation.
Level 2 constructs represent AWS resources with sane defaults and boilerplate provided internally so you do not need to worry about it when writing code.
Level 3 constructs are called patterns. These can create multiple resources at the same time and reduce the amount of code you need to write, as long as the pattern matches your requirements.
Terraform does not have the concept of constructs or levels and most terraform code closely resembles the level 2 constructs of CDK. Terraform modules allow you to create re-usable pieces of Terraform that contain many resources which resembles the level 3 constructs of CDK, but you have to write them yourself.
Documentation
The Terraform documentation for the AWS provider is comprehensive and covers every resource including arguments and outputs.
The AWS CDK documentation is not consistent at all. It is clear that different teams have created different CDK resources and documented them to different standards. This is further complicated by the construct levels system of CDK which means that the documentation for some resources is just a link to the CloudFormation documentation. This can be extremely frustrating as you have to context switch from Writing code in your preferred language to working with CloudFormation, in some cases having to guess how the API might work.
Community support
Sometimes you get stuck and find yourself googling for a solution. With Terraform the HCL language surprisingly works as a benefit because it is a single language used by all Terraform users.
With CDK you may have to make your search term a bit more generic and convert the answer from one programming language to another. With layer 1 constructs you might also need to convert the answer to a CloudFormation question to a CDK solution.
Looking up resources
There are differences between looking up resources with Terraform and CDK.
For example, with terraform you can lookup a Lambda function based on name
data "aws_lambda_function" "existing" {
function_name = var.function_name
}This is not possible with CDK and you need to look it up based on ARN instead.
PythonFunction.fromFunctionAttributes(this, 'lambda', { functionArn: `arn:aws:lambda:${this.region}:${this.account}:function:${functionName}` })Interestingly the ARN contains the function name which makes you wonder if Terraform is actually searching by ARN behind the scenes when you do a lookup based on name!
API Support
CDK is likely to support new APIs that AWS releases quicker than Terraform as they develop the APIs. It is likely that the teams that create and maintain services also create and maintain the APIs and CDK, which would explain the difference in code quality of CDK among the different services.
This may change with the release of the new AWS Cloud Control Provider which automatically keeps up to date using the AWS Cloud Control API.
Dependencies and versioning
Terraform providers are versioned using a string literal containing one or more conditions, which are separated by commas such as version = ">= 1.2.0, < 2.0.0".
The versioning of CDK dependencies/modules will depend slightly on the language that you are using. In our case we use TypeScript which means we install dependencies with npm . Unfortunately it seems that CDK modules do not follow semver which can lead to unexpected behaviour if that is what you are expecting. Every dependency needs to be exactly the same version (including minor version) to be sure that it will actually work because interfaces sometimes change in patch versions which goes against semver - this has led us down rabbit holes trying to work out why things don’t work, until we locked all the CDK dependencies to the same version. Here is an example of what I mean;
"dependencies": {
"@aws-cdk/aws-apigateway": "^1.120.0",
"@aws-cdk/aws-config": "^1.120.0",
"@aws-cdk/aws-ec2": "^1.120.0",
"@aws-cdk/aws-elasticloadbalancingv2": "^1.120.0",
"@aws-cdk/aws-elasticloadbalancingv2-targets": "^1.120.0",
"@aws-cdk/aws-iam": "^1.120.0",
"@aws-cdk/aws-lambda-python": "^1.120.0",
"@aws-cdk/aws-logs-destinations": "^1.120.0",
"@aws-cdk/aws-route53-targets": "^1.120.0",
"@aws-cdk/aws-ssm": "^1.120.0",
"@aws-cdk/aws-wafv2": "^1.120.0",
"@aws-cdk/core": "^1.120.0",
"@aws-cdk/custom-resources": "^1.120.0",
"source-map-support": "^0.5.16"
}This can be a bit annoying if you are not aware of the issue. It also shows another annoying thing about working with CDK; there can often be different versions of the API for the same thing!
In this case we are using the dependency @aws-cdk/aws-elasticloadbalancingv2 but there is also a @aws-cdk/aws-elasticloadbalancing - it is up to you to work out which one you need as this is often not well documented.
Code analysis
There are a variety of tools that can be used to analyse Terraform code. Examples include tfsec which checks for security risks, and tflint for linting.
With CDK you can make use of all of the tooling that is already established for your programming language such as intellisense, linting, and testing.
Extension
Sometimes you might need to do something that is not supported by a Terraform provider or CDK. In these cases you will need to write your own Terraform provider or create customer resources in CDK.
When it comes to creating a Terraform provider you will need to use Golang — if you are lucky enough to already be using Golang anyway this will not be a problem. If you do not use Golang it’s quite a burden to have to learn a new language and then learn how to create a Terraform provider in that language.
With CDK it is much easier to create custom resources and you can do it in the language that you are already using. Here is an example of a custom resource that we created in TypeScript;
const eni = new AwsCustomResource(this, `eni-${index}`, {
onCreate: {
service: 'EC2',
action: 'describeNetworkInterfaces',
parameters: {
'NetworkInterfaceIds': [eniId],
},
physicalResourceId: PhysicalResourceId.of(eniId),
},
policy: AwsCustomResourcePolicy.fromSdkCalls({ resources: AwsCustomResourcePolicy.ANY_RESOURCE })
});This custom resource allows you to look up the private IP addresses of ENIs of a VPC endpoint which is not currently possible via the provided CDK constructs. It is quite impressive that this can be achieved in only 11 lines of code (it could be 1 lined but wouldn’t be very readable!).
Conclusion
If you are only using AWS services, do not have prior experience with Terraform, and are using a programming language that is supported by AWS CDK; CDK is likely to be quicker to get started with. Additionally, if your infrastructure requirements follow a common pattern you may be able to make use of level 3 CDK constructs to reduce the amount of code you need to write.
Although CDK is faster to get started with, you might hit some roadblocks along the way as the documentation and community support is not as good as Terraform.
If you have more complex infrastructure requirements that span multiple cloud providers, or use tools for which there are existing Terraform providers then Terraform may be a better choice.
There are some cases where CDK can be beneficial such as those where you need to insert JSON into Terraform. HCL has very limited capabilities for constructing JSON programmatically, while this is extremely simple in a language such as TypeScript and can greatly reduce the amount of repetition.
After having used both tools I would now consider CDK for smaller projects and Terraform for projects that are larger with more complex requirements, however there is nothing stopping you from using both!
