Hidden Depths of Product Requirements

When a testing task comes in, the first thing we usually think about is what the requirements are. If I have to perform a test, I need to know the expectations in order to evaluate the extent to which my system under test (SUT) corresponds to them. The more requirements I have, the more test cases I can think of.

But what if I only get high-level requirements, a short description of the core feature’s functionality, or even worse — a Jira ticket with just the title and a totally empty description field. What to do in this case? I see 3 possible scenarios here:

1. send the task back to a task owner and ask to provide more details;
2. take what you have and do your best to test the product;
3. start on a requirements journey of your own.

The first two scenarios are quite feasible. It’s totally OK to request the feature owner to provide more information. And I for sure understand someone who would do their job based on the task assigned. However, if you want to be more than a person executing written steps, it’s your mission to gain enough knowledge to be able to create a test coverage that is able to reveal the majority of bugs hidden in the product.

A missing product requirement is a potential production bug. The cost of such a bug, found after the release, is about ten times bigger than if it were found at an early development stage.

If you want to discover more about the requirements, stay with me. Let’s examine the requirements together, learn where to look for them, and why QA should question each piece of information they receive.

The requirements are the set of expectations that SUT should meet. The requirements can be explicit — formally documented in a certain tracking system, or implicit — based on common knowledge about the usage, appearance, properties of a particular feature. Let’s get into details.

Explicit requirements represent the core idea of the feature. This is what we, testers, use to check the positive flow, to perform a sanity test cycle. If we’re talking about a business feature and not a technical task, explicit requirements are usually described by product managers as a User Story.

Example: “As a user, I want to add a product to a shopping cart.”

This simple use case gives me a clue about my main test case: add a product to cart, check that the product has appeared in cart. Then I can think about more cases like:

• add same product to cart;
• add several items of the same product to cart;
• add a few different products to cart;
• etc.

Let’s imagine that we delivered an online store website with an ‘add product to the shopping cart’ feature to the end-user. What if some of our users already have experience with the logic of online stores and expect our application to work in a similar manner? For instance:

• when adding a product to the cart, there is a cart icon component that shows how many products there are in the cart (we can call it ‘best practice’ for online store sites);
• what if there’s a law in our target country which forbids the sale of certain products to persons under 18 y.o. (this requirement comes from legal);
• it could be that we’re doing a redesign of some kind and our users expect the new product to support the ‘old’ feature, like cart preview (this requirement is called product history);
• and so on.

The set of requirements above was not documented, but they are all important for the product to be competitive, conform to legal requirements, and consistent with previous versions. These are the implicit requirements.

You can ask, how would QA know about these requirements? Where to look for them? Here are a few tips about possible sources:

Products History — does your product have a ‘history’? If yes, learn thoroughly the previous version(s) and pay attention to when the new product misses some of the ‘old’ features. That ‘old’ feature might be missing on purpose, but it’s a QA job to verify it.

Similar products inside your company — my product is an eCommerce platform, however, at Wix, there are a bunch of other verticals that also have the Checkout Flow. In this case a ‘vertical’ is a software product that solves business problems of a particular group of users, like store owners, fitness trainers, event managers, etc. What if each vertical has a different UI, different navigation, and some custom features implemented in the Checkout Flow? I bet that’ll drive your users crazy! Good user experience practices suggest that similar products should be predictable and do exactly what users would expect.

Similar products on the market — same as the previous, but instead of comparing similar products in your company, take a look at products of the same type and kind on the market.

Developer code / System Architecture — read the code, talk to developers, ask them to draw a model of the system, check what they expect from it. Sometimes you may find hidden features you would never find unless you knew exactly where to look. It could be error handling, limitations, big data-related flows, etc.

World wide known standards, regulations — Accessibility, GDPR, performance standards. Apply whatever is applicable to your product.

Analytics data — if you have access to the company’s analytics data, you have everything you need. There is no better source of info on your product and your users:

• Most popular browsers, operating systems, mobile devices;
• GEO breakdown;
• Product funnels.

User feedback — since our main goal is to satisfy our customers, we have to take care in listening to what they have to say. Take a look at their feedback: what are their main pain points, do they clearly understand the product, is there anything they’d like to add? This should give you some good ideas about what to consider as requirements.

Just be mindful of the fact that sometimes a user can complain about how a product behaves when what they are lacking is a specific feature. It’s like when you ask someone “why do you need a glass?”. People usually say that they need it to drink water. What if they don’t have a glass? They might say they will use their hands. So the reason they need a glass is to make it convenient to drink water, which means it doesn’t have to be a glass per se.

The last thing I want to talk about is that you shouldn’t perceive each requirement you get or find as the source of truth. Since we are testers, we should question our requirements the same way we question our product. Requirements may also contain bugs: misinterpretations, uncertainty, impaired integration with different product parts, conflicting conditions, etc.

Often problems with requirements arise when those come from different sources: product manager and technical lead, designer and developer, worldwide known practice and custom solutions.

The final set of requirements should represent a system of rules and conditions that clearly describes all product aspects and unequivocally states if certain product appearance or behavior is desirable or a bug.

As a conclusion.

Requirements represent people’s expectations about a given product. They are divided into explicit — formally documented in a certain tracking system, and implicit — based on common knowledge about the usage, appearance or properties of a particular feature.

It’s QA mission to gain as much knowledge about SUT as they can in order to create a test coverage which would be able to reveal the majority of bugs hidden in the product.

Here are some good tips on where to look for implicit requirements:

• Products History
• Similar products inside your company
• Similar products on the market
• Developer code / System Architecture
• World wide known standards, regulations
• Analytics data
• User feedback

Finally, each requirement you obtain should be questioned and tested the same way we test the product, because requirements may also contain bugs.

Start your requirements journey, I hope you will enjoy it!