Are you a QA, as in Question Asker?

Do you ask questions? Do you ask the right questions? Do you get the right answer back?

This year I was one of the judges in the regional phases of the Software Testing World Cup and I also had the pleasure to personally watch the finals.

The STWC is a 3 hour competition in which the testing teams have to design a testing strategy, test, report bugs and produce a final report. During the competition, the testers may ask questions to a product representative to fine-tune their focus.

Some of the questions asked were related to users’ most used browser, expected behavior in different devices and scope related questions. Do you ask those questions? They may guide you through decisions such as how much time should you dedicate to testing a particular scenario, device or browser, and thus, make your testing more efficient.

In the finals, the product tested was Zalando and the representatives were from business, quality and development teams. During the tests, an interesting question came up. One of the competitors asked what was the most fragile feature they had, the last one in production or the one they knew had an interesting problematic historic. It’s a perfectly pertinent and useful question on our everyday job. The answer was that it was a competition and they couldn’t answer that. But, truth to be told, everyone was hearing all the questions and answers, during the competition, so every question asked wasn’t an exclusive competitive advantage.

Although I believe the competition was the real reason why they didn’t answer the question, this is also a question hard to get the answer for other reasons. When asking a developer which of the modules she developed is the most fragile, you are asking in other words “where do you think you didn’t do an excellent job”, or “where do you think you did a bad job”, or “where do you think you’ve failed” or… I’ll stop here to try not to make things even worse. All the questions are probing for the same answer, but the wording is different and that matters a lot.

The testing job is often referred to as QA, usually meaning Quality Assurance. We can’t assure the quality, but instead we can provide information that will allow that. So, QA has gotten a new description among testers: Question Asker.

How to ask a questions matters a lot and that will determine if you’ll get the answer you’re looking for or not. Watch out for wording that can make the other person feel bad or exposed. And as you’ll often make this sort of questions, try to foster an environment in which asking and answering these questions is ok.

I’m currently reading a book that I already highly recommend: Thinking Fast and Slow by Daniel Kahneman. Learning a bit of cognitive psychology will give you very useful communication insights into getting the answers for your questions.

I keep imagining different types of questions that could have been asked in the competition. Personally, I like making business questions and taking insights from it.

“So, this is an e-commerce website that has clothing and accessories for everyone, women, men, kids, right? What are your best sellers?”

“What about your costumers? What’s the average customer profile?”

“How much time do your customers usually spend in the site?”

“What’s the average shopping profile? How much do they spend? How many items?”

“How do the customers usually get to the site? Facebook, Google, Twitter, Ads campaigns? TV, Radio? Newsletters?”

“What’s the engagement with the newsletters? Have you been able to reach lots of customers? How do they subscribe to it?”

“Do you have a feedback system implemented? How does it work?”

“And what kind of feedback do you usually get?”

And there goes plenty of other questions. So what do those have to do with your testing job? First of all, they are related to your job. You work for that company, it’s natural to expect that you want it to succeed, and it’s very positive to understand how everything works.

Now with the answers, I get to those questions I might be able to:

  • draft initial personas to drive my testing.
  • determine the importance of the newsletter subscription feature. I know it’s not the main feature, obviously, but what if it’s the biggest attractor of customers to your website? What if it’s responsible for the highest number of sales? The answers will either make me ask more questions regarding it or move on to another subject/feature.
  • by exploring the feedback, get to know about bugs and bad experiences users had such as not finding a product, price mismatches and so on.

So, focus on the answer you want to get and explore all the questions you have to make the other feel comfortable to give you all the information.

Let’s keep talking about asking right the right questions on Twitter @SarahPimentel.