Punchcard: imagining the future of cloud programming.

When I first heard of the AWS Cloud Development Kit (CDK), I was immediately inspired by its incredible vision. Using general-purpose languages to model AWS infrastructure is a game changer — it elegantly abstracts away the low-level JSON and YAML of CloudFormation (CFN) by generating it with beautiful class libraries. I hate writing clunky and error-prone CFN templates, but I love programming. The CDK keeps my focus centered in modular code instead of monolithic JSON files, so development feels organic and fun instead of something I do once and hope to never touch again (like a janky CFN template). Amazing!

It is so clearly a next-gen productivity tool for AWS — be sure to check out their official documentation and announcement blog.

This is the beginning of a blog series describing Punchcard, a high-level framework I built on top of the CDK to imagine “futuristic” infrastructure-as-code (IaC).

The name stands as a reminder that our current tech may one day feel as ancient as punch-cards. You know, the days when this was considered a code repository:

Deck of punch-cards. IaC in the good old days (source).

Fork that!

I think IaC has the potential to increase productivity by many orders of magnitude — just consider how expressive the CDK already is even at a low level:

new sqs.Queue(stack, 'MyQueue');

It’s like literally saying to AWS, “gimme a ,” and then sitting back and enjoying the show — it’s kind of magical!

By reducing CFN templates to an implementation detail, the CDK creates an awesome opportunity to make use of modern programming techniques to build high-level and general infrastructure. Configuration files can’t match the expressive power of code, so let’s get crazy with it.

I’m personally super excited for constructs like a ““ or even an outlandish “”! I wonder, how would the world change if it were possible to instantiate a whole company’s architecture in one line?

Punchcard

Punchcard adds to the vision by unifying infrastructure code with runtime code, meaning you can both declare resources and implement logic within one node.js application. AWS resources are thought of as generic, type-safe objects — DynamoDB Tables are like a ; SNS Topics, SQS Queues, and Kinesis Streams feel like an ; and a Lambda Function is akin to a – like the standard library of a programming language.

To demonstrate what problem it solves, I’ll build a simple example with the vanilla CDK and then show how Punchcard improves the developer experience. Specifically, I’ll create a CloudFormation Stack containing a node.js Lambda Function which publishes notifications to an SNS Topic.

In the vanilla CDK, we require two files: and .

app.ts — the CDK infrastructure.

The CDK application code creates a single containing an and .

To publish SNS notifications from within the Function, we grant to its and add the to the variables. The runtime code can then look up that value, initialize an SNS , and begin sending notifications to our new Topic. The runtime implementation is configured by declaring the language , entry-point , and a path to the on disk.

To deploy the application to CloudFormation, simply run :

CDK application with a single containing an and

This is a huge improvement over raw CFN templates. CDK constructs simplify the process of creating and connecting AWS resources, and (because it’s code) anyone in the community can share their own classes, functions, and modules containing infrastructure solutions. Boom! The paradigm shift has emerged – you just can’t beat the utility of code.

handler.js — the Function’s implementation.

But, we still have to write the runtime logic for actually sending notifications to the Topic.

And, it’s pretty ordinary.

The script (below) looks up the from the environment variables, throws an error if it doesn’t exist, creates an SNS for making API requests, and finally serializes and sends JSON notifications to SNS.

Function implementation, showing boiler-plate code for publishing to an SNS Topic.

As you can see, the runtime code is tightly coupled to the infrastructure. It must always discover the ARNs and names of resources or else it can’t call AWS APIs; it must also take care to only send valid data to a service or else downstream consumers will break. This is fragile and repetitive — if you get any of this wrong, expect to find out after deploying (when your Function is throwing runtime errors).

Also, how can you build high-order abstractions for constructs without also generalizing the runtime behavior? They go together. By referencing this opaque file, we’ve lost the rich context of our infrastructure and are instead stuck managing boiler-plate. Bummer.

index.ts — the entire Punchcard application.

It can’t be stressed enough how innovative the CDK is, but let’s take things a step further — to a higher level of abstraction with Punchcard. We’ll think of constructs as ordinary data structures (like in-memory programming) so you can create, use and extend objects with the same code. They’ll also be type-safe to enable the compiler and IDE to detect and alert us of mistakes in real-time, saving cycles waiting for deployments and testing a broken application in AWS.

That Lambda to SNS example is now simpler and safer, contained in a single file, :

Lambda ⇒ SNS with Punchcard.

Let’s walk through the code.

A Punchcard application is the same as a CDK application except for one minor detail. You have to the as :

const app = new cdk.App();
export default app;

I’ll explore this further in another post, but just know that this convention serves as the entry-point to your runtime code, and effectively bootstraps the Punchcard abstraction.

Next — we create a and define the of data it accepts.

const topic = new Topic(stack, 'Topic', {
type: struct({
key: string(),
count: integer(),
timestamp
})
});

That represents a JSON object, such as:

{
"key": "some key",
"count": 1,
"timestamp": "2019-07-30T23:45:00.000Z"
}

Then, when creating the , we declare that it on the . Declaring a dependency automatically grants IAM permissions and sets environment variables containing details such as the Topic’s ARN.

new Function(stack, 'MyFunction', {
depends: topic,
// etc.
});

The end result is that a client instance representing the construct is automatically instantiated and passed into the function at runtime. This client encapsulates the logic of looking up the , creating the SNS , and safely serializing rich JavaScript objects to JSON — the annoying stuff.

new Function(stack, 'MyFunction', {
depends: topic,
handle: async (event, topic) => {
await topic.publish({
key: 'some key',
count: 1,
timestamp: new Date()
});
}
});

Notice how the field accepts a object, leaving it up to the framework to serialize timestamps to a .

To clarify, the passed in to is of type:

Topic<{
key: string;
count: number;
timestamp: Date;
}>

And, its function therefore has this signature:

public publish(notification: {
key: string;
count: number;
timestamp: Date;
}): Promise<AWS.SNS.PublishResponse>;

This type-safe mapping between infrastructure and runtime code simplifies the implementation of the function, reducing it to a high-level and statically checked closure:

await topic.publish({
key: 'some key',
count: 1,
timestamp: new Date()
});

If you mess up, your code will not compile. Nice!

In this post, I introduced the philosophy behind Punchcard and showed off some of the basics, such as runtime code, dependencies, and the type-safety provided by its embedded type system. There’s plenty I didn’t talk about though, including testing, security boundaries, streams (and other DSLs), and how the code is bundled.

Follow me on twitter to get notified of upcoming blog posts. As a teaser, I’ll be diving more into the DSLs — explaining how this code configures a Lambda Function to process SNS notifications:

topic.forEach(notification => {})

If I’ve managed to pique your curiosity, you should also check out the Punchcard GitHub repository and take it for a spin. If you run into trouble, I’d love to hear from you on gitter.

What awesome ideas do you have for generic cloud infrastructure?

Programmer. I’m obsessed with modeling AWS infrastructure with type-safe and generic programming. (link: https://github.com/sam-goodwin/punchcard)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store