Some Takeaways from Let’s Test 2016

I had the great honor and privilege to speak at Let’s Test in Sweden this year. In the time since the conference, I’ve had a chance to reflect on the presentations and see what’s resonated with me back at work.

The Only Thing it Really Depends on is Money — Scott Barber

I wish all of my colleagues from the advertising agency where I work could have been in the room for this discussion. Every contract our company signs, every feature we develop, every time we choose a severity level for a bug, our decision comes down to the value we’re providing a client. With this in mind, software testers should aim to:

• Deliver what the client wanted, not what they asked for.
• Provide informative testing metrics. When a client asks for a metric that isn’t useful, provide them with a different metric that still provides enough information to evaluate progress and make decisions. One useful question is “What shouldn’t I do if I were to demo this to the client today?”
• See what people are using the most in the product. Make those things better.
• Bundle the cost of testing with the cost of the product. Nobody wants to pay separately for testing.
• It doesn’t matter if a product is fast if nobody buys it.

But the one thing I think about every time that I write a bug or ping a colleague is this:

Testing as an isolated activity has no inherent value. Nobody pays for testing. People pay for the information testing provides.

It’s useless if I find a bug and I only tell the other testers, or the developers don’t understand the bug I wrote, or the severity isn’t properly evaluated. The bug fix won’t find its way back to the client to make the product better. To keep the most important issues top of mind, schedule a periodic bug triaging meeting with the product owner.

These other thoughts that came up during the discussion weren’t directly related to the topic of the presentation:

• Use the few extra minutes after scrum to define some terms with your product owner.
• For job satisfaction, you want to be a trusted advisor. You want important people asking you interesting questions.

Context Eats Process for Breakfast — Patrick Prill

Patrick walked us through a couple exercises about describing your morning breakfast routine:

1. Describe the perfect cup of coffee.
2. Draw how you make toast.

Like some other participants, I had no idea what made a perfect cup of coffee. My perfect cup of coffee is tea. Here’s my version of toast:

1. Break off a piece of baguette. 2. Put it in the toaster oven for five minutes when it’s a darker color. 3. Serve on a plate. 4. Butter is too hard to draw.

I drew my toast with a few steps involving a baguette and a toaster oven. Most others had more steps that involved money, stores, butter, sliced bread, and a conventional toaster.

Patrick used our descriptions and drawings as a springboard to distinguish tacit (unstated, assumed) from explicit (stated, known) knowledge. By asking questions to clarify the assignment and our audience, we could have made the task more explicit and come closer to the tacit expectations.

Just as with breakfast, testers need to be able to question assumptions and accurately describe their processes in the course of their work like when they:

• Describe steps to reproduce a bug.
• Write a training manual or other technical documentation.
• Code a test script.

Providing too little context confuses the audience. Providing too much context distracts them. Finding a balance is key.

The Human Factor: Exploring Social Engineering — Dan Billing

Dan’s presentation changed how I want to test and generally live my life. I thought I was comfortable with how much personal information I shared, both on the internet and in person. Dan had us try to find out as much as we could about someone else in the room by Googling them. And showering them with flattery. I was able to get a lot out of people. It was probably enough information to guess a password.

Before the end of the presentation, I was completely paranoid about what every salesperson, person around the office, or person near my calendar or phone could be learning about me. But Dan shared a deeply personal story with us. He trusted us. So after destroying my faith in humanity, Dan helped restore it. Thanks Dan!

Dan shared his slides and more information about social engineering here.

Gaining Consciousness — Fiona Charles

Fiona’s closing keynote centered around a paired exercise. One person was the business owner, explaining their context to the tester. The other person was a tester, asking questions about the context. After a few minutes of brainstorming, the pair compared notes.

Ideally, the business owner would have given the tester all the right details to answer the tester’s questions. But no pair had a complete matching set of information. Overwhelmingly:

• Testers did not ask about the business model.
• Business owners didn’t provide information about the users or politics of the people on the project.

So testers and the business owners rarely start on the same page. To get more in sync, Fiona recommended having a periodic sanity check of your test strategy with the business owner. We need a space to say “Here’s what I’m thinking about. What do you think?”

Other presentations

I’m still thinking and talking about all the presentations I attended. Thanks to Rob Sabourin, Anne DeSimone, Bolette Stubbe Teglbjærg, Andreas Cederholm, Christopher Lebond, Helena Jeret-Mäe, Erik Brickarp, Rob Bowyer, Martin Hynie, and Damian Synadinos. I’m enormously grateful to all the presenters at Let’s Test, the conference for giving me the opportunity to speak, and everyone I talked to over those three days. Learning more about your journeys is what keeps me in this profession.