Why user testing must be a team activity

Designers do user testing, well at least they should… Everybody knows this, it’s written down, Moses put it on a stone tablet, we get it. Blog over, have 5 minutes of your life back.

The problem is that all too frequently I see user testing being treated as an exclusive activity for designers alone. Yes, it is likely that the majority of user testing will be driven by designers, they are closest to the prototypes we often want to test, they often have the most experience and insight into testing methods and they are the most likely to implement a change (in the early stages). So it makes sense to cut out the middle man and get them to facilitate.

However a good user testing plan should engage users right from inception through to delivery, which gives ample chance for everyone to have a go.

‘But I’m busy’, ‘I’ve got loads of meetings’, ’You’re better at it than me’, ‘I want to code’, ‘Leave me alone Tom’ are just some of the initial responses when trying to convince people to come along.

Many of these comments may in fact be true, so why should they bother? Well lets have a look shall we?

Seeing your work in action

No matter what your role is, the hope is that you contribute to the overall project and therefore the product in some way. Yet it’s only the crayon-wielding Sketch monkeys who get to see the product in the hand of users. Why should they have all the fun?

Seeing a product being used in context will help you to build empathy with a user, which you can apply to work later down the line. Furthermore, you have the option to engage, challenge and query at the time. Every individual will interpret a testing session in a different way, but it’s only by being there that you’ll have the ability to interact with the user and try to tease out additional insights, that others may have missed.

Finally, if you helped to design/build the product, you deserve the chance to see it in action. We don’t join a system/service design company to sit behind the scenes and never see it in action. Would you work on a film and not go and see it?

Getting a better understanding of your users

Personas, empathy maps, as is scenario maps…all very very useful tools. But none of them are actually real. They are imitations, best guesses and approximations of reality. There is no substitute for sitting with a user in their natural habitat. Would you rather see a picture of a lion or one in Africa (other mammals are available to describe users…). When observing a user you will get to see their natural behaviour, with no filter*. They are more likely to say what they would naturally say, do what they would naturally do and make the mistakes they would normally make. I believe the further a user is from their natural habitat, the less reflective of reality any results of testing will be.”

Move someone to a lab setting for testing = 20% less reflective

Report findings of ethnographic study in meeting = 36% less reflective

Conduct usability study over the phone = 83% less reflective

(percentages have been made up a tiny bit, but you get the point)

*Observation obviously has an impact on behaviour in some cases, but when it comes to user testing, it’s the lesser of a number of evils.

So, get on the ground with your users. Breath their air, see what they see. Become one with their environment.

Be a digital Bear Grylls.

Prioritising and acting as a team

“EXCUSE ME, sorry to interrupt but I did some user testing and we need to remove those 3 stories to incorporate my changes, thanks”.

“But they are criti….”

“NO, I HAVE SPOKEN”!

All user testing findings need to be acted upon in some capacity. That can be everything from deciding to ignore them, to making them the sole focus of the team. This all depends on their severity. Severity ranking of findings should be a team activity, similar to estimating. After all, it’s all work that will require time from the team. Therefore the team needs to decide how severe a finding is, how much effort is required and when it needs to be sorted out.

Engaging the whole team throughout the process will stop shock and disruption from happening.

The alternative is improvements getting shoved to the bottom of the backlog, or improvements being shoved into a sprint because it’s a critical issue, even though you’re halfway through another story. Do either of these situations sound familiar?

Different perspectives on findings

We are all very different people. We have different opinions, different skills, different wants and needs. We also observe the world around us in different ways. These differences can really come into their own in a user testing session.

It’s all too easy for findings to simply represent the interpretations of 1 or 2 people within the team. This will inherently create some form of unintended bias. Exposing a wide range of individuals with different perspectives to user testing, can help to iron this out. It will also unlock new findings which might only been seen by somebody with a different eye.

Fixing shit

User testing always ends up being a great opportunity to flush out bugs and issues (providing you are testing with live code). Often a user will interact with a system in a way that you hadn’t envisaged and throw a metaphorical spanner right into your works (ouch).

The benefit of a cross functional testing team is actually two fold here.

Firstly designers trust designers and devs trust devs. Now if something is fundamentally broken, everyone should get on board with the fix. But what if it’s something subjective, something that varies between user testing groups, an opportunity for A/B testing perhaps…?

Chances are you might get more resistance to change. But in a cross functional group, everyone can champion the need for the change to their respective peers. “I saw it, it’s an issue. Let’s try changing it”.

Secondly there’s the fundamentals of actually fixing the issue.

It can be difficult during testing to appropriately establish the source of a bug. It can be even harder to replicate or describe this to the rest of the team the following day. Having a dev at hand can help to solve this, as they are far more likely to be able to spot the source of the issue.

I’ve even had it where the dev does a fix locally whilst I carry on testing. If it’s a serious bug, this can make the later testing so much more productive. They can then go back to the team the next day and accurately describe the issue, or just push their fix into develop.

Summary

I’ll keep this short, you’ve been here long enough. User testing is a magical time where you get to suck information and improvements out of the people that matter. Arguably there is nothing more important when it comes to system development. As a result everybody should be involved in it.

Get as many pairs of eyes, as many opinions and as many perspectives as you can. Everything you’re getting from the testing is solid gold, so make sure that you’re gleaning as much as you can from it, from as many people as possible.