Documentation is terrible
If there is such a thing as an universal constant in development, it’s this: for a given repository or body of code, the documentation will be, at best, out of date. At worst it will be misleading, or just entirely missing. The reasons for this are many; ranging from pressing deadlines, to the fact that writing documentation is simply… not fun. This, we can all agree, is terrible. Documentation is how we, as code consumers and code editors, get insight into what a given project is about, what is going on in the head of the people who started it, how to set the project up locally, or how to query the API. In short, documentation is perhaps the single most important part of a project, and one that is constantly and repeatedly neglected. There are of course exceptions (e.g. the React Documentation is among the best out there), but for a given project, odds are, the documentation is not all it could be.
Documentation is a big topic. There are lots of things that need to be documented, from APIs, to internal tools, and depending on the audience and type of documentation, needs will vary wildly. This blog post is not going to comprehensively cover everything you need to know, but hopefully, it will help you write better documentation, whether ‘you’ is your team, your open source project, or just you as a developer.
What you need to know
Documentation matters: If you have ever had the despair of looking at a blank README for a complex project and trying to figure out what it does or how to set it up, you already know this. If you’ve ever tried to use an API only to discover that the queries do not work, because someone forgot to include that you need a header with your auth token, you already know this. Less anecdotally, your documentation is how people know how to use the complex system you’ve poured thousands of hours into. Without it, you are depriving your user-base or fellow coworkers of the most rudimentary ability to use your tools. Instead you’ve ensured that the only way the code base can be used is via an archaic type of tribal knowledge, and this will come to haunt you.
Documentation is good for you: From a purely utilitarian point of view, if you write good documentation, and encourage others to do the same, then maybe when you return to a project a year later, you can just open the README or the wiki and get to work, instead of spending a week trying to figure out that the C bindings on this ruby gem only work with a specific version of MySQL that you can only install by compiling from source. If you write good documentation, people will use your stuff. Which is one of the best feelings in the world. Documentation forces you to think from other perspectives, not just your own, which ultimately results in better, more usable software. As developers, we spend more time reading code than writing it, so it behooves us to make sure we have the tools to read the code as effectively as possible.
There is no silver bullet: Knowing that documentation is bad (almost) everywhere raises a question that I’m sure we’ve all thought about once or twice, at least, or, if you’re obsessed with usability like I am, every other night. “How do we fix this?”. Surely there must be a tool, that if included in a given project would fix everything? A git hook that requires the documentation to be up to date, or an auto-documenting tool. Unfortunately, we can learn from the proliferation of unit testing culture here: no amount of tools will make developers do something they haven’t bought into.
Documentation is culture: For developers to write documentation, they need to be persuaded that documentation is important, that it’s worth the drudgery, and that they, personally will benefit from it. There’s no way around this. Culture here refers to the discipline within a group of projects, OSS, commercial, or even the single repos you maintain on your own. If there is no constant habit of keeping and updating documentation and of writing documentation whenever a project starts, and if everyone doesn’t buy in then, inevitably, the documentation will suffer. If no one is making the effort, then it simply seems easier to not do it at all. To have good documentation, you need a culture of documentation. I could write a whole separate blog on how you foster a culture like this, but here we can borrow from the lessons learned by organizations espousing TDD; you need to get buy in from everyone, and set up procedures and expectations that documentation is a deliverable that is part of your code.
Documentation is part of the work: It would be totally unthinkable to submit a code change that hasn’t been tested locally, or without knowing the extent of the change. The same must apply to documentation. A code feature isn’t complete until the documentation has been updated. That means for a new project, there is an effective README, for an existing project the existing README has been reviewed, and the existing documentation matches what now exists in the repo. For professional organizations, this means you must budget this into your time estimates. I usually allocate about an hour of documentation for every four hours of work, but like with unit tests, this can become time consuming. If documentation matters to you as an organization (and it should), then you will allocate time for it.
Keep it up to date: I talked about this above but I’m repeating myself here because this is so important. Keep your documentation up to date. Having outdated documentation is like keeping failing tests, it just creates noise, makes development harder for developers, and ensures your code is unusable. We’ve all run into a library before where the API documentation is out of date. Days and weeks of developer time have been wasted on outdated auth processes or systems where no one wrote a simple one line update for a public facing API. Worse is the case where the documentation is so outdated in some places, but not in others!
Centralize your documentation: You know what’s as bad as missing or outdated documentation? Documentation no one can find. Or documentation that is spread across three wikis, five different types of google docs, eighteen repos, and a handwritten scroll. You should have a central hub of documentation, and a README in each repo. The README should include basic information, and the central hub should have all the extended knowledge. It doesn’t matter what you pick, as long as everyone is using the same thing, and there’s an organizational pattern to find the information you need.
Write for your audience: Your documentation has an audience. Sometimes that’s other developers, other times its shareholders, QA teams, system administrators, or a whole slew of different people. Depending on the audience, you want to organize your thoughts rather differently. Before you write your documentation, think about what that audience is, and what’s important to them, and lead with that. Make it easy for them to find what they need. Be as technical or basic as you need to be, but above all, give them the tools to use your tools.
Documentation doesn’t replace reading the code: If you’re writing documentation for other developers, you’re not expected to write a paragraph for every function. You’re supposed to cover what the reader doesn’t have insight into, so the purpose of your software, an overview of how its achieved, what they need to do to set it up, or authorize. Give them the tools they need to understand the code, to use the infrastructure, but don’t do their job for them.
Tests are not documentation: Neither are comments. Code is not self documenting. Documentation is about giving the readers the tools to use your project, and to understand why it exists. To point them in the right direction and set them up. It should be convenient and accessible. Tests are great for more nitty-gritty specifics, but they do not replace the need to give an overview of why the code exists, or to call out how to install dependencies, or to give all the valuable context that lets us reason about problems.
So what?
“Well that’s just dandy” I can hear him, the imaginary hyper critical 10x ninja rock-star developer in my head. “How exactly do you write ‘good’ documentation then?” The sneer in his voice is palpable. in a way he’s right, these are all nice words, but they don’t mean much without practical advice. In the interest, however, of keeping this document legible, I’m going to focus on just one type of documentation.
Writing good READMEs
A README can be your only documentation, but it doesn’t have to be. I suggest that if you have a long, complex API with lots of examples that is constantly evolving, to have a form of documentation separate from the README that users can easily search. The assumption is that the README is located in a repo someone intends to clone, is for a program someone intends to install, or a library they plan to use. That said, I’ve seen very good READMEs that included extensive details on their API and leveraged markdown anchors to make it navigable and user friendly. Generally, most good READMEs consist of the following parts:
An introduction: I recommend writing this before you even write a single line of code. Short and to the point, the first sentence should say what your software does. “This provides an API for counting dogs in the continental united states” or “A tool for randomly deleting your AWS credentials so you can see 403 errors”. It should be one line, and as complex or as simple as necessary. It should let a potential user know what they’re getting into. It also focuses you on what the heck it is you’re doing. Later, when you’re thinking of potential features, you have a written statement of what your software does, and you can figure out if your feature is in scope. Then a couple lines as to why, if its merited. Goals without purpose are fine, but as developers we instinctively seek the why, having it written down makes it easier for a potential reader to understand. Optionally, this can be a good spot to talk about “how” in a high level way. Maybe you’re leveraging machine learning, maybe you generate your results by using a giant regex to parse html, whatever it is, it’s helpful to know the broad strokes of how it’s being accomplished. Remember, short and to the point. The longer it is, the less likely a developer eager to save whole minutes is going to read it, no matter how many hours it may save down the line. Link the extended documentation here somewhere. Preferably in big, bold letters, with a hyperlink and the word ‘documentation’ so that a desperate user looking for the link can find it quickly.
Setup Instructions: How do I, a random user who hasn’t the faintest idea of what a Javascript is, get your fancy technology on my computer? These come in two types: for developers, and for users. One is focused around installing for minimum use, the other around making code changes to the specific repo, but the process of generating them is the same. Go through the steps. Start with a clean virtual machine/coworkers computer/spare laptop, and make your project reach a workable state. We often forget all the crap we’ve installed for various projects (Oh yeah, i have Postgres 4.1 installed for that other project!), and that can make us miss basic install steps. By replicating a clean environment, we can ensure we get all the steps. Do this every time you add a new dependency. If you’re really feeling generous, you can use this process to generate a script users can just run to set everything up. Include ways in which the user can verify that the setup succeeded. This doesn’t have to be a tutorial on how to get all the dependencies setup, but it’s helpful to say “npm is required, here’s a link to their install instructions if you don’t yet have it, we also assume you’re running OSX version later than blah”
Examples: Have a couple quickstart examples, hopefully expounded upon in the documentation. Have unit tests that run these examples so you always know they work. They should show basic usage because, inevitably, it’s less simple than you think. After they’ve done the setup, most developers will just find an example from here to get their basic functionality, so make it easy for them. Include the basic cases, and link the extended documentation again so they can go read it if they need a more complex use.
Latest Changes: Somewhere in the bottom of the document, there should be a link to a changelog, and the last change that was made. This is invaluable for people updating their version of your library/tool/FooBarFactory, since it lets them know at a glance what areas were recently touched. Even for internal tooling this is inordinately valuable, because if something breaks, you have an area to look to (and yes, you could look at the commit history, but how often do we see a commit called ‘commit’ that touches 50 files?). Having a changelog that can serve as a reference is invaluable.
Troubleshooting: This section will start blank, but I recommend making it whenever you make the README. Fill it out whenever someone has an issue setting up the project, or running it, or pushing a change. This is where you write things like “if both awk and awk-grep are installed the library will fail to initialize due to excessive fluxing of the capacitors, to fix it, run sudo rm -rf“. This avoids the issue where you know someone has had the problem before, but can’t quite remember the fix or the stack overflow question with the fix. It should be last because this really only matters when errors are happening, and the user will ctrl-f to it by searching a keyword
Does this seem like a lot of work? It is. Good documentation is a commitment. Its difficult, and time needs to be budgeted for it. It’s also a habit. Once you get used to writing down all the setup steps and of making sure troubleshooting stuff goes in the README, it becomes second nature. That doesn’t mean it’s not annoying, but it’s something we do because we recognize the value of it, ultimately, documentation has the potential to make our life so much easier.
For everyone’s convenience, I’ve created an example template.
