User testing in a box*

The traditional approach

If you are reading this, you’re probably familiar with user testing, which at its simplest is watching people use websites — finding problems and informing designs.

The classic way of doing this is in a lab at a design or research agency. The users and the client team go to the agency, and observers — including the client team — watch the testing take place in a separate room.

The problem is that everyone involved has to travel to the lab — it means at least a day out of the office for your team, and half a day for each user. This inevitably means it’s almost impossible to get everyone in the client’s team involved, negating one of the great benefits of this kind of testing.

Recent research has shown the one thing that improves the quality of a solution more than any other is to increase the amount of exposure your team has to actual users:

“The solution? Exposure hours. The number of hours each team member is exposed directly to real users interacting with the team’s designs or the team’s competitor’s designs. There is a direct correlation between this exposure and the improvements we see in the designs that team produces.”

— Jared M. Spool

And it’s equally hard for participants — requiring them to travel to where the testing takes place often rules out busy, disabled, elderly or geographically isolated users, removing whole demographics from the testing.

Although I’ve been testing with users for about 15 years, and I’ve observed many of these sessions run (very well) by other agencies, I’ve never had a lab of my very own, despite many requests. So like most people, I’ve had to improvise. Guerilla testing, where a designer heads out on their own with a laptop and a notepad, is fast and cheap, and can’t be underestimated — but it loses that vital client involvement.

So what’s the alternative?

At my last agency, we developed a kind of ad-hoc lab solution using the built in Mac screen sharing and a couple of meeting rooms. It’s a good solution. It works, but it did mean the user has to use a mac (which never goes down well), and the observers couldn’t see the user’s face or where they clicked. And everyone still had to come to us.

“Where’s the @ sign?”

— every PC user on a mac, ever.

I’d been working on-site at a client, coaching them in the ways of User Experience, and using it in an Agile process, and despairing of ever getting the right people in a room for a day away from their work. So I set up the same system for a day of testing at their office — changing mac based screen sharing to Skype, which gave us a picture-in-picture feed from the webcam.

With the simple addition of coffee, biscuits, a friendly account manager wandering the corridors, and a sign on the door, we managed to get not just the core team in for most sessions, but many more people from all over the company dropping in for a look. The day ended with the Marketing Director popping in for 10 minutes, staying for an hour, then declaring:

“We’re never launching anything again without doing this!”

Within 3 months they had hired their own full time UX designer, which for me is the ultimate compliment — even if it’s not so great for my employer…

A lab in a box

And so, our user testing lab in a box was born! 2 laptops, a TV or projector, a decent network and 2 rooms is all you need. And the 2 rooms don’t even have to be in the same place any more. (more on that in another post). You can set up a testing room in a different place and Skype it to a client’s offices.

And now, there’s more…

Since then, here at Hive we’ve developed the system further. Google hangouts lets you broadcast the testing to a private youtube channel with on screen chat, so the number of people and locations that can get involved is now limitless. It works on tablets and phones so we can test them just as easily. And you get a permanent record without third party screen recording tools.

Even more…

And until very recently, a key bit of lab based technology, eye tracking, was impossible with this set up. Now though, I have a new toy:

That is an eye tracker that can almost fit in a small box and costs no more than a tablet — rather than the usual £10,000 upwards!

So we now have a full user testing lab in a box* that can come to you.

And it costs less than a tenth of a traditional set up in hardware, and doesn’t require us to have 2 rooms always dedicated to it.

* Well, actually, my rucksack, but that’s not as catchy.

Want to know more? Visit ux.hiveit.co.uk to learn more about Hive IT’s UX research, testing and design services. You can also download our UX Menu, our way of making UX Services easier to digest.