AWS CDK for Platform Engineers — The Mindset
Author: Artem Volkov
The first article is in an “AWS CDK for Platform Engineers ” series:
The Mindset
CDK as a second language
Over the past year, as the Platform Team Lead at a large company, I’ve been developing a platform using AWS CDK. The organization is currently transitioning from a traditional data center to the AWS cloud. Concurrently with this shift, there’s been a requirement to execute numerous projects for the various businesses within the group. Our team is made up of software developers, many of whom lack experience in cloud development, and Site Reliability Engineering (SRE) professionals who have strong skills in AWS and Terraform.
The content of these articles is aimed at Cloud engineers, such as SREs and DevOps professionals, who are skilled in Terraform but are encountering difficulties when adopting CDK. My approach will be to examine the issue from the standpoint of Platform engineering.
It is presumed that you can find and study the AWS CDK Workshop, so basic CDK details are not covered here.
The series of articles include:
- The Mindset
- CDK as a second language
- How to choose Programming Language
- Platform design
- Build reusable components
- Golden Path templates
The Mindset
A short history of IaC tools
Around a decade ago, my role involved using Python and Boto to automate the creation of AWS resources. The results were impressive during a client presentation. I managed to set up a complete test environment, including external dependencies, through code rather than manual clicks. However, the trade-offs regarding the maintainability of the solution weren’t evident during the demo. The system was composed of several Python classes utilizing all the object-oriented programming techniques I was familiar with and eager to apply. At that time, CloudFormation was the prevalent method for infrastructure management, known for its rigid, YAML-based configurations, until Terraform came along and offered a new approach.
Oh, it was a declarative magic. During that time Terraform and its HCL become an industry standard. Readability and extensibility allow us to manage not only Cloud resources but any platforms that provide an API.
The emerging practice of utilizing actual coding languages for managing infrastructure seems to be a never-ending cycle. However, one might wonder why there’s a shift towards this approach and what necessitates the adoption of a fresh Infrastructure as Code (IaC) instrument by professionals in Infrastructure, Site Reliability Engineering, DevOps, and Platform engineering when Terraform continues to be an ideal solution.
The answer is simple — CDK is not an IaC tool.
Common Misunderstandings and Concerns About CDK
I am a huge Terraform fan, but also a pragmatic engineer. I think that we have to choose the right tool for the right job. So let’s demystify some statements.
CDK is IaC
We can divide the architecture of any IaC tool into three parts:
- User Interface. A way how you define an Infrastructure as Code.
- Engine. The core software element executes your code definitions, manages the state, and calls Cloud Provider API.
- Target API. It could be a Cloud Provider or any SDK of the service you want to manage.
Let’s map popular tools into these definitions.
As you can see, CDK is just a templating tool similar to Helm for Kubernetes. So we have to treat it only as a User interface to an IaC tool. It could be CloudFormation (AWS CDK), Terraform (CDK TF), or Kubernetes (k8sCDK).
I need to learn programming as an Infra engineer.
It’s always a good idea. I’ve had programming experience before, but I can’t compare it to a regular software engineer skill. You know it’s like you have added Python and Go to your CV, but in reality, you can create some script with Python and StackOverflow copy-paste.
You have to learn some basic concepts of programming language and most likely Object Oriented Programming. But it’s just a tiny part of what a regular software engineer skillset. Some variables, conditions, loops, classes, interfaces. inheritance, package management. The great news is that you will have a lot of opportunities to learn them hands-on during the infrastructure development.
CDK will allow developers to replace Cloud Engineers
Yes and No. We will talk about it more in the use case section.
It is hard for developers to write Terraform because developers can’t learn HCL. But because they have less infrastructure/cloud knowledge. They in general invest more time in learning business domains and programming languages, frameworks, etc.
So specialization still exists. When some software engineers could specialize in fintech or healthcare domains, SRE is investing time into Infrastructure and Cloud domains. So for complex systems, we still need an AWS expert to do the job.
But on another hand if it make sense for a developer to ask each time an SRE engineer makes a change in infrastructure? I am sure that it’s not necessary and even expensive for business.
Everything feels so uncomfortable
Just trust me it’s 80% a skill gap. I’ve spent 9 years writing Terraform code, and everything that works not the same looks stupid. Just give it time. I’ll help to map Terraform skills on CDK concepts in the next article.
The other 20% is a question of the maturity of a tool. It’s still young, but development is surprisingly fast.
CloudFormation is slow
It’s really slow. Sometimes it’s annoying for an engineer, but it compensates from a different side.
For big projects, CDK could be a glue that allows Dev and Ops to speak the same language. True DevOps culture. This synergy between personnel can lead to significant advantages for the company.
My previous experience with Boto3 was terrible
The AWS CDK goes beyond a simple SDK; it’s a framework designed to steer you in crafting your infrastructure as code. However, it’s important to avoid making the code unnecessarily complex. Just because you have access to the full suite of features in a programming language doesn’t mean you should use them all. Aim for simplicity and straightforwardness in your code.
One more time about user interfaces
First of all, we have to remember that CDK is not an IaC tool, but a User Interface. The most comfortable for software developers. Not always for you! We can quickly map favorite interfaces on each role:
- Infra/SRE/DevOps/Platform Engineer likes Terraform.
- Software Engineers or architects like CDK.
- Business analytics, Product Owner like Web.
Just keep in mind that as a Platform Engineer, you are choosing the User Interface for someone else. So you don’t have to like it, you Platform User should.
CDK Use cases
Serverless
In my view, the most outstanding advantage of the CDK is present. It enables software developers to swiftly develop applications by utilizing a single programming language for cloud services, the backend, and, if using TypeScript, the frontend as well. The capability to construct and deploy Lambda functions and Fargate containers, execute integration tests, and carry out deployments is truly remarkable.
We are historically on CloudFormation
I’m confident in your robust knowledge of CloudFormation and your skill in resolving its occasionally peculiar issues. The Cloud Development Kit might offer you adaptability and significant reusability for infrastructure elements.
Platform Engineering without Kubernetes
If your primary orchestrator is ECS and you have various Product teams employing diverse tech stacks, the CDK could be a viable option. It enables the integration of your organizational patterns into reusable templates and libraries.
For intricate AWS setups involving stateful services such as RDS, sophisticated networking, EKS clusters, or integrating AWS with external services like Keycloak or CloudFlare, Terraform is the advisable choice. It offers a reliable and secure solution for essential infrastructure components.
The decision between using CDK and Terraform can be determined by the level of AWS knowledge needed and the type of user seeking an Infrastructure as Code (IaC) platform.
Conclusion
CDK remains a relatively new tool designed to provide a user-friendly IaC platform for software developers. It is well-suited for serverless development and Platform Engineering tasks. Infrastructure Engineers need to adapt their approach to focus on usability and collaborative efforts within their teams. For intricate infrastructure configurations, Terraform continues to be the preferred choice.
