My “Docs as Code” journey #1
I am a french IT specialist that has been working for big french companies on some on their most critical pieces of software for 15 years. I worked as a developer, trained my teams to embrace devops and now I am working as an Architect to maintain a tool chain. This tool chain is being migrated to the cloud and will go through a 1500% volumetry increase in the next 2–3 years.
I’m starting a series of articles to describe how I manage the documentation of the projects I work on and how I changed my mind on how to manage it. Along the way, I will point how this has been beneficial for myself, my team, and my clients.
Documentation at work
The major pain point I encounter everyday is the lack of good documentation. A good documentation is up to date, accessible to its public, easily maintainable, and enjoyable to read.
Documentation is usually written in files. They can be hosted on a shared file system, Sharepoint, GSuite or any other document hosting solution. Documents may have several forms (document, spreadsheet, presentation, video, image, …) and are often duplicated to trace the history of each and keep every final version of a document.
From my past experiences, most of the documentation is created and maintained by developers or IT tech guys, that are not specially thrilled to write it and may not be specialist with the tools to put it down. It may be difficult to apprehend, maintain and may become obsolete fast as people start to stop spending time to update it.
Teams or companies follows guidelines to structure these documents and even the guidelines are not known by everyone. It happened more than once to ask for help to find what I was looking for. This is a mess for a new team member or a neophyte of the organization of the documents.
At last, sometimes, people were keeping the last version or working locally on a document. It breaks the referentiality of the repository solution chosen, and prevent the team to access the last version of a document.
This had to change.
First baby steps
A few years back we started using Gitlab, which brings a nice feature that is Markdown rendering. It allows to write documentation in markdown format that is rendered in HTML when viewing the file using some default layout / theming. Github and Azure also has this feature.
Markdown is a lightweight markup language for creating formatted text using a plain-text editor. Created in 2004, having a stable specification since 2014, this language focus on readability and can be read as is by any tech guy without any effort required or any heavy training to understand how it works.
Like Github, if a folder contains a README.md file, it automatically renders this file when browsing the repository folder, very convenient to explain what the files do or aims to. As on a website, you can link to other readme files or to other technical ressources (images, conf files, code snippets and so on).
Having the documentation as close to the code as possible has several advantages:
- Writer integrates better with development teams. Like the devops trends, people writing the documentation are closer to the team and it creates more exchange between all the profiles in the service (dev team, support, test team, etc…).
- The documentation is part of the delivery and developers can write drafts of it before the final version and as it is managed like code, it can be reviewed and discussed.
- Also, a feature can be blocked if the documentation is not properly done and validated. As a result, people are more committed toward quality and are committed in maintaining the documentation up to date.
We started using it on projects that were lacking documentation and the ability to directly point to code files or configuration file or tools files (How to build / package the app? How to install the app locally? etc…) was really a plus. I noticed that the dev team was more involved in the process, pointing out that these files were our new referential.
Our focus was to describe the technical part of the applications, how the configuration works, what are the process of each service, how to package the app, how to deploy it, etc… It was nice but the lack of graphical documentation started to rise up.
How can we describe the architecture of our components, how to describe the process and interactions without to rely on a wall of text ?
Improvement our text based documentation
We started to use PlantUML to achieve this behavior.
PlantUML is an open-source tool allowing users to create diagrams from a plain text language. It has plugins in all IDE that helps out to create schemes and diagrams. As it follows a “Schema As Code” principle, the images are stored in text files that are rendered when browsing them. Also storing images as text is more efficient when you want to version it in Git, as it’s not a binary file, it only stores the text diff between 2 versions.
PlantUML is the way to go to define the UML diagrams our documentation requires.
Exploring what plantUML is able to do, I came accros C4, not the explosive version nor the car. The C4 model is a convenient way to define the architecture of an application with a series of schemes that decompose it in container and components. I find it a nice tool to get into software architecture on an application level. Check out the official website, it’s worth a try.
Alongside the code we now had described all the internal process using UML and the externals interactions of all our deployed components and applications.
Adding even more stuff
As part of our developer actions, we use other tools to create integration tests (Jmeter, SoapUI, …) or manage some calls collections (Postman, Intellij HTTP requests, …). The configuration files of those tools found their way inside the repository, alongside with some readme to help new users set up the tools and work with the files of course ;).
And then the light shined even brighter. The team started to create template of components and document how to use them, explained how some helpful bash scripts were working, added some test data and document how to use them and the expected result in the app, etc…
Share the knowledge outside the team
It was a nice enhancement and all the team wanted to be involved in the maintenance of our newly created documentation. While it was a success internally, we started to use it outside and extends its audience. Documentation describing configuration files set up, deployment files, UML schemes of applications interactions are used by the team but can be interesting as well for people outside of organization. Support team, DevOps team, Architects ans POs have to benefit from our work.
Having the documentation intricated within our code, it is not easily accessible to everyone, especially when you must navigate in folders and work with branches / tags to retrieve the correct documentation / version. We have to get a step further and share our knowledge.
Gitlab has a Pages feature to create a static website and host it. We used Jekyll to generate this website based on the markdown files. That way there is a single source of information for the documentation.
We stumble upon some troubles that highlights the limitation of our current solution:
- Sometimes it is a pain to find what you are looking for if you are not familiar with our own structure / organization.
- Our Gitlab Pages authentication is bound to the Gitlab authentication + group membership. So people outside the tech guys requires a licence seat to browse the documentation and there is an initial set up to do the first time you access the documentation.
- Markdown image insertion of generated PlantUML files is not supported by default by the Gitlab Flavoured Markdown. It results in different rendering if viewing the documentation inside Gitlab or with the static generated site.
- we have some duplicated documents, that are almost the same between different projects.
- The edition of the documentation for non tech people is a concern. If an architect wants to edit the architecture schemas, he may not be familiarized with the usage of Git in the team. For pure functional people or manager, they are unable to do it by themselves.
- We are unable to view an old version of the documentation on the generated site.
These are not huge problems, but we are not 100% satisfied having them.
What’s next ?
In a few weeks, we succeeded to have a better documentation, involved the team to keep it up to date and change the approach that doc must be done in word, ppt and such. Out transition works smoothly but we want to go bigger and mitigate all those pain points. That’s what we did, and this will be treated in the part #2 of the series !
In the next article I will focus on how we fixed all the issues and how we have an industrialized and centralized documentation.