Diagrams as Code for Infrastructure as Code

Mario Fernández
Sep 9, 2020 · 4 min read
Image for post
Image for post

If you’re working with infrastructure, you’re undoubtedly leveraging infrastructure as code, right? I sure hope so. Otherwise, start now. I’ll be waiting.

A natural evolution might be creating a graphical representation of your infrastructure. Diagrams as code if you will. It is not a new idea. This technique appeared in the tech radar back in 2015. You can hark back to that period where people thought that generating code from UML diagrams was a sensible idea.

Let’s forget that ever happened! Instead, let’s think about using visualization to understand complex architectures. And maintaining those artifacts up to date while we’re at it. I’m going to talk about a tool that builds on top of Graphviz called Diagrams.

What is diagrams?

A tool with a very generic name, that’s for sure. If you check its website, it’s described as:

Diagrams lets you draw the cloud system architecture in Python code.

Pretty straight-forward. Installing it is a one-liner.

pip3 install diagrams

We’re going to create infrastructure diagrams with Python. We’ll commit both the image and the code used to generate them so that everybody in the team can make changes. Let’s get coding, err drawing. I have two examples for you.

Alerting workflow

Let’s say you have a workflow where you publish alarms to an SNS Topic. That topic is processed by a lambda function, which transforms the data and writes it back to Cloudwatch. Through an event rule, the right alerts reach our event bus.

So much text for such a little thing. I’m surprised you’re still paying attention. Let me hit you with a fancy diagram.

Image for post
Image for post

If you check the code, you’ll notice how simple it is. There are three elements.

  • The nodes, which are single components. It has a recognizable icon and a name.
  • The edges, which are the connections between components. They can be directed or undirected.
  • The clusters, which group nodes logically.

And there is not much else. The diagram is built with this snippet.

Notice how quickly you can get a decent overview of your system with a just bit of Python.

Network diagrams

Networking seems to be uniquely suited for this approach. While researching VPC endpoints, I realized that a clear drawing aids understanding significantly.

Let’s say we are connecting to a Kubernetes cluster because that’s what we all do these days. We’re routing a bunch of different domains to a VPC endpoint. The cluster resides in a different account, so we use an endpoint service to make it available. Add a few more routes, and the whole thing becomes a tangled mess, much like the pile of cables behind your desk. That is until you see this diagram.

Image for post
Image for post

Better, isn’t it? What about the code used to generate it?

The amount of code used to represent it has grown compared to the previous example. Luckily, using Python gives us access to functions, comprehensions, and other tools to manage this complexity.

The curious case of the diagram that could not be updated

That’s how you use Diagrams. Is it really valuable? I certainly like the pretty icons. Apart from that, it shines in enabling visual documentation to evolve.

I’ve tried to digitalize technical diagrams before. I really tried. Sketch, sketchboard, or even just taking a picture of a hand-drawn diagram. It works until you need to update it, and you don’t have the source. Perhaps it was done by somebody else, who preferred a completely different tool. I’ve often seen a project’s documentation get more and more out of date because nobody can update the damn diagrams. If it’s just source code, your chances get a lot better.

Conclusion

Diagrams is a neat, albeit limited tool. You can’t add much more than what I’ve shown. While that is constraining, it can protect you from yourself. Overly complicated diagrams do more harm than good. If you are representing the system in its entirety, why not check the code directly? The point of abstraction is to make it simpler to understand by omitting some of the details.

In summary, it’s a convenient way to bring clarity into your impenetrable READMEs, and you’ll be able to update the images as your code evolves.

Originally published at https://hceris.com on September 9, 2020.

The Startup

Medium's largest active publication, followed by +752K people. Follow to join our community.

Mario Fernández

Written by

I develop software for a living. Then I go home and I continue reading about software, because I just cannot get enough. Nowadays I work for ThoughtWorks.

The Startup

Medium's largest active publication, followed by +752K people. Follow to join our community.

Mario Fernández

Written by

I develop software for a living. Then I go home and I continue reading about software, because I just cannot get enough. Nowadays I work for ThoughtWorks.

The Startup

Medium's largest active publication, followed by +752K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store