Read me; Build me!

What if your README was also your task runner, or your build script?

Lee Machin
Jul 21, 2017 · 5 min read

Bear with me on this one, or in fact, dog with me if you will. Dog is a tool written by some fantastic friends/ex-colleagues of mine at Typeform. It is a task runner that allows you to declare your commands in a YAML file and, for all intents and purposes is a more generic alternative to Make. You don’t have to use a Dogfile to wrap your various build and deployment processes, but considering the audience it is likely you will do exactly that. This is what we do at Happiness Works right now: it wraps a couple of dozen commands related to getting a Docker setup up and running, simulating the CircleCI environment, deploying, running migrations, opening consoles in production and all that stuff. It is not complete but it’s satisfying to build your own CLI tool from a YAML file, without resorting to the hacks you need to do that with Make.

There is one thing I’ve found lacking with these tools and the same applies to Grunt, Gulp, Rake, Make, Jake, Cake, William Blake, Ee-ake and whatever other tool exists to serve the same purpose using the language it was built in, and that is the understanding of what these tools aim to abstract.

A new hire unfamiliar with Docker and their employer’s choice of CI pipeline isn’t going to learn much about it by executing bundle exec cap production deploy or make build. The inner workings of those scripts can be intimidating and they tend to build up a lot of cruft over the years as new things are piled on but old things still have to be supported. In many cases this complexity is unwarranted as you are deferring the work to another tool already. You probably don’t need to fully understand what make build does when it is setting up an entire pipeline to convert your C code to an executable—you are rarely going to do that by hand because you have completely transcended the point where gcc dostuff.c -o dostuff is going to work for you. If your tooling, however, demands you npm install -g grunt grunt-contrib-*... before you can run grunt build, which only actually runs docker-compose build -f docker-compose.local.yml, then you are far better learning the tool being wrapped.

We use READMEs for documentation, and GitHub (as far as I know), popularised the concept of using Markdown for your READMEs. They extended the format with Github Flavoured Markdown which you can now refer to as CommonMark, a specification for Markdown parsers. Through that extension you get fenced code blocks which allow you to render code in monospace text and also apply syntax highlighting relevant to the language contained within it.

This makes Markdown appropriate for literate programming. On the simplest level you can extract the code from fenced code blocks and ignore everything else. It’s just an inversion of how a typical file of code is parsed: instead of having a special syntax for comments you have a special syntax to start and stop the parser. But what if you took it to another level and made the entire file significant?

Imagine a Markdown file roughly like this:

If you are deliberate and consistent about the organisation of this content and how it is parsed, you can create a build tool that serves the dual purpose of being both executable and readable. You can leverage existing tooling to generate a table of contents from this that the parser will ignore, but will still provide a way for the user to navigate the document. They can see commands and subcommands, a description that serves as help text, and extra content that is there primarily for the benefit of the reader. In essence, a specific structure and subset of Markdown is used to build a CLI and the rest is documentation for it.

I imagine that, if you parsed this file in such a way, you would get a command suite that would print out full help text if you asked it, and run commands like zozz parrot simon says hello and zozz compiler push. You could download a tool that parses a markdown file on the fly, much in the same way existing tools do, or you could run it through a CI pipeline and get a standalone binary that is updated whenever that markdown file changes.

This may not have much value to experienced developers who are used to building these systems up from scratch, or maintaining existing tooling. My own motivation behind this idea was to encourage juniors or new hires to learn the code behind the abstraction for simpler tasks. If you’re moving to container-based infrastructure, or you’re setting up something using Vagrant, you become much more productive if you know how those work beyond running make start-vm.

You can do this already by just adding comments to those files, but somewhere along the line in recent years colour scheme authors have decided that code comments should have low contrast and be as low-key as possible. This is ironic because comments in a file should generally be exceptional: the fact a comment exists above a line means it should be important, because it is one line of documentation in a whole file full of executable code.

Suppressing comments through the colour scheme is a response to developers using comments so much, and so ineffectively, that they have to be made as unobtrusive as possible so you can still see the code. I imagine this is not just because of aspiring programmers writing } # end for loop or # returns true if arg is present. all over their code to help them understand what is going on, but because of documentation systems that, due to lack of first-class language support (dynamic typing, no concept of docstrings, or ambiguity in the case of languages like C++), parse their special format out of code comments. Suddenly comments are no longer about pure human documentation, they are just hacking comments into more code only this time you’re programming a documentation generator. Even that is frustrating because a one line function will carry a lot of commentary baggage: one line that says /* , one or more lines of documentation code, and a final */. No wonder you don’t want to see them!

Basically, you’re not going to open a file, see that it is completely full of comments, and then read it from start to finish. You’re just going to ignore most of it and try and find the thing that matters. But with something like a README the expectation is already set that it was written to educate.

More simply, how often do you keep the documentation of your build tasks up to date with the build tasks themselves? That requires you to update two files that no compiler can verify for you. You can delete a task and forget to update the documentation, you can rename it, or make it do something different. You will not always remember to update the textual references to those tasks.

This is something I am trying to solve with zozzfozzle, which is also a good reason to learn some Rust. If you can use your README or another markdown file to power a suite of commonly used commands, the documentation and the code are automatically in sync. You will plainly see if they’re not because it will be reflected in the CLI output and the help text. Maybe it doesn’t prevent issues, but makes them incredibly easy to report (this task claims to do x but y happens!).

)
Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade