Overestimated value of test documentation

Photo by Wesley Tingey on Unsplash

Disclaimer: This article is aimed towards people involved in the development of software used in web technologies. The author expresses his own point of view and does not claim it to be the only one acceptable and derives from personal experience. Naturally, there are other business fields, processes, and technologies with different requirements, priorities, and approaches.

The subject of test documentation is a well-described subject in various sources. Every new QA who starts getting involved in this activity learns some theory of what is being used to keep the process of testing tracked and structured. There are different approaches, but eventually, they all define the same types of test documentation [source 1, source 2].

Let me remind us of the famous ISTQB glossary for definitions:

Test Plan:

• describes what test objectives to achieve and why;
• sets a schedule for achieving them;
• coordinates testing activities;

Test Case:

• a set of preconditions, inputs, actions, expected results and postconditions;
• basically what we do step-by-step in each case and what we should expect to happen (auth.)

Traceability Matrix:

• the table that correlates two selected entities (e.g., requirements and test cases).
• can be used for tracing back and forth the links of one entity to the other to be able to track general test coverage.

Test Report:

• documentation summarizing test activities and results
• a comparison of expected and actual results (passed and failed tests).

Quite a lot of information, in general, can be summarized to a set of steps which an average QA should follow:

1. Create a test plan in the initial project’s phase.
2. Create test cases for the current stage of the project and follow them during test execution.
3. Provide a test report that demonstrates failed and passed tests at the end of the phase (iteration, stage, milestone, etc).
4. We should keep in mind that such a report is aimed to provide a reflection on the project’s status at some moment of time. Of course, there will be fixed bugs and passed tests afterward. Tests will be performed not even once.
5. Maintain a traceability matrix to show what’s already been covered and what remains undone.

And now let’s try to capture a bigger picture of our imagined project. Our brave QA is a part of a brave team which should include at least one developer, probably a designer as well, and of course a manager (sometimes people try PM-less projects but they end up nowhere. Believe me, there has to be someone to control processes). Team members make their contributions according to role and skills. But they also have somewhat different goals. Yes — they all want the project to deliver success to the business (i.e. to gain profits), but:

• The manager wants project growth and efficiency (more users, or more features, eventually leading to more money). In the modern world, it means to deliver more and faster, preferring speed over the project’s “cleanliness”.
• The developer wants the project’s architecture to be stable and maintainable. That contradicts the PM’s desire to add more features as quickly as possible.
• The QA wants tests to be green, bugs to be fixed, projects to be understandable, and thus testable.

Considering such a tug-o-war between team members, can we keep the quality level and standards as high as possible at all? Let’s take a quick look at what the QAs are ‘armed’ with. On the image below you can see names of abstract and concrete things that we use in testing.

Red-colored words relate to what is required by known project management methodologies and ISTQB. Blue words represent things that are needed for good agile testing. And words in pink are showing us actions or features that good testers require in general.

The inventory of modern software tester

In modern web development, the competition is big. The market is fast-pacing and customers’ needs change quite frequently. Web software development companies choose AGILE ways. That’s why blue items from the image truly matter for QAs. That’s a powerful set of tools to organize testing in your project.

Of course, it’s a big plus to be able to understand how things work, to communicate with other team members to learn more from them, and to have a good view on such abstract features like usability for example.

Red items, essentially, are not important in agile development, since they are meant to break the agile and turn in into something hard (even though stable) with almost no space for maneuvers.

So having dropped the unimportant things, it’s much more pleasant to test projects no matter how unpredictable their growth is. Here are some additional tips on how to better cope with agile projects:

Optimize your understanding of testing documentation.

1. Don’t waste time on test plans, as well as on the traceability matrix. You may end up rather more confused and lost in a long read of changelogs.
2. Checklist matters. Of course, you need some starting point. When the project starts or a new feature is added — start with a brief checklist in order not to miss anything related to regression testing and keep an eye on some complicated system parts. But again — consider it a draft and don’t waste time on beautifying that. You are most likely to be the only one to read it.
3. No one reads test docs. When you deliver the project — the end customer wants it to work. A long list of tests with test statuses won’t be read, people are too busy with their businesses.
4. Automate — your test-automation should become your checklist for later tests. You will always be able to see tests’ names in CI they run in. You should be able to read tests’ names to understand what test checks in general in order to reproduce the test manually if need be.

Know your system.

1. If you know it so deep you understand how the project is deployed, what are IP addresses of servers and what libraries are used, and how — you’re most likely a developer. But if you are a tester — you need to have a general overview of systems components and their ties. Thus you will be able to ask questions and think of cases to make cross-component tests.
2. Better knowledge of the project structure helps to find more regression test cases and add them to automation.

Memory matters.

1. Human memory has its limits. But those limits are not low. And, most importantly, they are expandable. Invest your time to optimize brain activity and learn how to structure any information you have to memorize it organically.
2. You don’t need to know everything — but you need to know where and how to search for things you need. Notes, emails, JIRA tickets are a great way to point out some project (design or usability) issues even before they are created. You can always look them up and show that it was in your input before, but for some reason not handled.
3. Learn best practices — market competitors, dependent products, even governments, directly and indirectly, claim new approaches. The best of them become used in many projects. Eventually, it may have an impact on users’ views and ways of thinking. As a QA you need to be aware of it.

With that in mind, you have more chances to make the project go smoothly. Time is valuable so should be invested properly. Exploratory testing becomes a big deal since it does not require paperwork and relates more to actual testing with UI interactions / API calls / checking of responses etc. Modern web projects require modern QAs. Good luck!

For more engineering updates and insights:

• Follow us on: Twitter | Facebook | LinkedIn
• Visit us on GitHub
• Subscribe to our monthly newsletter
• Subscribe to our YouTube channel
• Follow our Medium publication