Raising the standards of our documentation: Doc as code and static API
Efficient knowledge sharing requires high quality deliverables. Content should be structured and consistent in order to be easily digestible. Common tools such as corporate wikis provide great support, but they do not impose any structure or governance for your documentation.
As coordinator of an internal unit whose mission is to foster knowledge about the web development domain, I maintain a beautiful Confluence space where all the content is nicely organized. However, I wasn’t really satisfied by the current strategy: I wanted something more structured, especially because I wanted to move towards a Doc as Code approach (as some big tech companies such as Microsoft or Zalando). After some digging, I decided to move everything to git and to define all the content as YAML or Markdown.
As an example, let’s consider the Brownbag (i.e. talks or presentations given during lunch time) section. In Confluence, each brownbag had a dedicated page where you can find information such as title, date, abstract, etc. With the new strategy, we have one YAML file per brownbag with the following structure:
People that would like to give a new brownbag need to create a similar YAML file and open a new pull request to the git repository. By combining some scripts to check the validity of the data with some git pre commit hooks we can prevent malformed or incomplete information to be unintentionally pushed.
The next step would be to publish the content somewhere: I have a Nginx instance running on our corporate network so I decided to upload everything and serve them as static assets. However, the end result wasn’t extremely attractive: people were able to access the brownbags’ data via a boring URL directly pointing to the static file.
Wouldn’t be awesome to have a REST API instead? Of course it would, but usually this implies maintaining a web server and a DB. Nowadays the approach of statically generating web pages during build time and serve them as static assets is gaining lot of traction and therefore we decided to use the same strategy in our situation.
First, we need to convert all YAML files into JSON.
After that, we moved everything into an appropriate folder structure that will represent the final path:
We are approaching an API like experience, but users still need to add the file extensions to the path and there is no endpoint that returns the list of all brownbags:
Fixing the latter point is easy, we just need to merge all JSON files into a single array:
For solving the first one, however, we need to modify our Nginx configuration. The idea is to leverage its try_filesdirective and to check the existence of an index.json file or a corresponding .jsonfile before fallbacking into a 404:
Sweet! With just one Nginx instance and a bunch of static files we can offer a real REST API-like experience to our users! People just need to touch the current endpoints and the server will return the JSON object of the corresponding brownbag:
JSON is a really suitable format for transfer data object but YAML is more human-readable. Therefore, we decided to push also the original files to our Nginx server and make them available via the same interface. Usually, the AcceptHTTP header is used to express what kind of MIME type the client is able to understand and consequently we decided to use it to select the desired format. The IANA MYME type registry doesn’t have an entry for YAML files and therefore it’s considered standard to prefix the type with x-, resulting in the application/x-yamlvalue.
Thanks to Nginx mapdirective we are able to select the desired file extension and return the correct content to the user. Of course, when no AcceptHTTP header is present, we fallback to JSON:
At this point, we just need to update our try_filesdirective to support the new $file_ext variable:
We are now able to query the data with different formats! All we have to do is to define the AcceptHTTP header when executing the request:
The next step would be to directly serve a markdown representation of the brownbag description. Since we are essentially web devs, we implemented some scripts using the google/zx tool, leveraging both the power of bash and JavaScript. Our script takes as input the YAML file of a brownbag and returns a string representing a markdown document:
Once also these files are pushed to Nginx, we update the mapdirective supporting the new text/markdownMIME type:
Thanks to this new feature, there are some interesting use cases that are now unblocked. First, it becomes extremely easy to GET the content of a brownbag in markdown format, transform it to HTML and append it to a web page. Second, since Confluence is the default documenting tools used inside our organization, we use the Confluence REST API to publish the same content into the dedicated space.
But not only: thanks to document converter tools such as Pandoc we can now pipe the returned markdown and generate a multitude of different formats. For instance:
Going crazy, we could also render the markdown file directly into the terminal by transform it to HTML first and piping it to a text browser such as Lynx:
We are extremely satisfied about this new strategy to manage our documentation. By following a Doc as Code approach, we are able to maintain a structured content enforcing the quality by exploiting the same tools and approaches we use during development without being locked to any vendor. Moreover, by exposing the content via a HTTP API, the integration with third parties’ tools and systems becomes extremely easy, fostering even more the adoption of our content inside our organization.
This approach allows us to raise the bar of our technical documentation, by treating it as an essential product of our system with the same high standard we have with respect to user experience, performance, interoperability and accessibility.
