The Benefits of Technical Documentation: When and Why to Use It

How often do you hear “we need complete documentation”? I’ve heard this a lot during job interviews. When asked, what comes to mind first? Perhaps, some of these: User story, SMART, INVEST, consistent, full, and testable. But understanding the criteria in theory and applying them in practice are two different things. How can you measure your documentation? When do you need it, and why? Let’s try figure it out.

Hi, my name is Aleksandr, I’m working in the Payments Platform team. Before Exness, I worked in a fintech product team. I’ll share my knowledge, examples from practices, and my thoughts, which I hope can help you.

Documentation vs Agile

The Agile framework is used almost everywhere nowadays, even if it’s not suitable for the occasion. So you’re likely working with some variation of it.

The Agile manifesto claims: “Working software over comprehensive documentation”.

But how do you know when your documentation is complete enough? When does it become ‘comprehensive’?

This quote is sometimes used as an excuse to avoid writing any documentation, instead scribbling a few things and never updating them. Do you think this is the correct way? Maybe it can work for a startup or a small company, but in other cases, you likely need to have up-to-date documentation.

Well, let’s now discuss what exact documentation you need.

When do you need technical documentation?

As the company grows, you will have new teams, new integration, new features, validations, rules, etc. If you don’t have quality and complete documentation, sooner or later you will face many problems which can affect your product negatively.

This is where the system analyst comes in, whose responsibility among others is to keep our documentation up-to-date. In my previous fintech job our product-oriented team knew everything about our product: we had documentation for every process, function, API, event, etc. Since the Platform-based product communicates with many other products, there are multiple processes and teams delivering changes to the Platform code, making the documentation process much more complicated.

Upon receiving a task for a new feature/implementation/bug fixing and gathering all stakeholder requirements, you ask yourself the following questions:

I know WHAT I need to get from a business perspective, but HOW can it be implemented? What will the side effects be? What tests should I write from scratch, modify, or delete?

Getting the answers requires you to know everything about your Platform processes, which seems almost impossible to achieve. This is where documentation comes in.

Good documentation helps you with:

• Improving your own knowledge of a product
• Finding potential problems
• Finding existing processes that need to be optimized and improved

Documentation also helps:

• Newcomers to get to know your product better
• Product/Project managers with MVP, scope, and creating dependencies map
• External teams to consume your data or integrate with your processes

What documentation do you need?

As a Platform team, we have many services, components, integrations, and data flows. Unfortunately, not everything is covered by documentation, but we’re trying to improve this. Basically, we can split the documentation based on purpose: external and internal.

External documentation

When you have multiple services interacting with each other, you will probably face data integration from external sources, or when external services (teams) need your data.

From the external team’s perspective, the main piece of documentation they need from you (as a team/service/product), is a contract’s specification. It doesn’t matter how they consume your data (API, Kafka, SQL, files), the main goal is to know what data comes, when it comes (or doesn’t), how to get/update it, and other integration cases.

To prepare such specifications, you have to describe how the data from your internal services transforms, what business rules are applied, and data mapping on your database tables.

In my practice one of the hardest formats for integration was ETL (Extract Transform Load): when you need to gather data from various sources (Extract), combine them and apply business logic and rules (Transform), and provide an external system (Load). As you can imagine, each step has to be strongly defined and described step by step.

I can’t say that API and Kafka are much easier, but usually they provide data as-is, with maybe a few additional filters.

Let’s see what the structure for API documentation can look like:

If your endpoint is just a ‘select from table’, then you can easily use Swagger. Otherwise you probably need to have the following information for your method:

• Name + method (POST api/{…})
• Description
• Input parameters
• Authorization
• Validators
• Output parameters
• Response codes
• Main flow + exceptional flow (where the logic of this method applies)

Of course, the word external doesn’t mean that your team doesn’t need this documentation as well.

Internal documentation

The next step is describing your core logic, which implements all basic flows, scenarios, data flows, etc. This helps everyone understand any possible problem in the current processes. It doesn’t matter if you have only 1 service which just stores some data, or a huge system with lots of services, integrations, and external dependencies — good documentation is always needed. Trust me, to complete this you’ll have to create lots of different documents about your product: functions, user scenarios, API methods, services, etc.

In my experience, I faced situations when there was no documentation and all knowledge was stored in a single developer’s mind. Just imagine what can happen if he quits. With documentation you won’t have such problems.

I still don’t see any advantage of such documentation, can you provide examples of when it really helped you?

Let’s discuss and draw a detailed comparison of situations where you have documentation vs when it’s absent.

Onboarding a new teammate

It doesn’t matter if you have a new developer, QA, or analyst — they all need to have a strong vision of your team’s services responsibility.

With documentation you can put it all on display using pictures, diagrams and even just text, which helps to get in touch with logic much better and can be re-read at any time. Without it, you have to rely on your own memory, which comes with the risk of missing important details.

New integration with your service

Detailed external documentation can be given to another team to work with and ask related questions when needed. You’ll still have to communicate with them, but most of the information will be readily available to them in your documentation.

Refactoring, decomposition, getting rid of legacy

Occasionally the team needs to make changes to their components due a variety of reasons: old technology or bad architecture that can’t handle increased loading, etc. To initiate and complete such an activity, you need deep knowledge of all aspects of your functionality. Without documentation it could be a real nightmare when after a few iterations you realize that you totally forgot about one main approach and have to start everything from scratch. You may have one great team member who has all the knowledge, but, as I mentioned before, it’s very hard to keep all aspects and minute details in your mind.

With documentation, you already have a scope with all necessary information, and it’s much easier to create a roadmap for your activity.

New features

It’s easier to challenge your new feature’s design/architecture when you have complete documentation because you can find all processes/flows affected by the new feature, and check how it can impact them. For the same reasons it’s easier to create a great test coverage for your services when everything is documented — especially if you have lots of cases and they’re all displayed in a diagram, since visualization is easier to understand and remember.

What if we can’t just dedicate time to it?

Of course, documentation isn’t something you can do in a few hours. If you don’t have technical writers or system analysts, there are several options:

1. Ask your developers to use automation tools to create it (there are solutions for auto-documentation based on code) — for the first steps this should suffice

• A great example is a Swagger integration which helps your team to automatically provide information about your public APIs to other teams (is a part of external documentation)
• Also for DB tables you can export table structure, so all your tables will be presented where necessary (and you can add additional details)

2. Delegate this activity to all your team members, create a single pattern, and use it universally.

• For example, those who write tests should do it with readable names and properties, so everyone will understand any scenario (such tests can serve as a User Scenario/Use Case)
• A developer working on a task should leave detailed comments on what he did and why in the task as a description, or in the code (then you can also use it in step 2).

3. Don’t write any documentation (then why are you still reading this?)

Where to begin when creating documentation?

The best option is to start from small pieces and move to bigger ones, like describing the glossary with base terms. After that you can describe storage for your data (DB tables, excel files, etc). Step by step your technical documentation will grow, enabling you to describe your business logic, rules and finally the processes.

If you need to describe core logic immediately, then start with determining the most problematic area. What cases usually lead to problems with clients, integration or other main points for your team? This will help you understand what exactly you need to reflect in the documentation.

How do we keep new or outdated documentation up-to-date?

There are different ways of writing documentation for:

• new features that have a totally new flow and don’t interact with your existing ones
• new features that change your current behavior
• bug fixes
• improvements to current processes

Ideally, documentation for all those cases should be updated after you release any changes to production. In that way, your ‘AS IS’ from documentation will correspond to your real behavior. This way helps you avoid rewriting because when you update it at the beginning of the development process, some parts can be changed by a developer, forcing you to update your update.

In the next article, we will cover the documentation types needed for different scenarios, the process of creating ‘TO BE’ documentation and some related processes, like ‘review of functional requirements’, so stay tuned for more!

Conclusion

To understand if you need technical documentation, you should answer the following questions:

• What stage is my company/product at? Is it the beginning of a startup or an enterprise solution?
• Does our solution have any dependencies? Do we plan to have them in the future?
• Do we plan to improve our product and add new functionalities?

Also, there are many factors that help you understand what kind of documentation you need right now:

• FAQ for support
• Readme for clients
• Swagger for external teams, etc.

To help you with prioritizing you should understand what kind you need right now.

Technical documentation can be external or internal:

• External helps other teams interact with your services
• Internal helps understand how your processes work and what business logic and rules are applied

Technical documentation is the responsibility of the whole team, so it can be done by a single role or delegated to all team members.

If you don’t have any documentation and you want to start creating it, please follow these steps and stay tuned for more articles on the matter.