“Imagine you are…”
Musings on User Testing
User Testing is something that User Researchers do to help inform decisions. There are a variety of tests, but I’m going to focus on one called “navigation test” or sometimes “tree jacking”. The basics of it are that you are presented with something like this:
and asked to move around the accordion to indicate where you think a piece of information might live within the architecture.
I just did one of those and noticed the questions were almost all prefaced with asking me to imagine something — like I wanted a specific kind of something, or had just traveled to a distant country and wanted to know if this store carried this thing, and what if I was a dieter, or what if I didn’t like soda?
At first the questions were difficult, and so I had to explore the majority of the navigation. What was presented to me didn’t match what the question was asking me to find — at least in my mind.
However, questions further down the test became much easier, since I had already hunt and pecked earlier. I wondered — would fresh eyes have been able to answer these later questions as quickly as I had? Are these later questions actually getting valuable data?
I thought to myself — I wonder what the results might look like if, instead of asking 10 people a lot of questions that cover all types of situations, User Researchers just asked them to work through one realistic session, and move on. Just be one person.
It’s definitely valuable to user test and I absolutely believe that when we can’t get in front of real users, asking others to have a little imagination is the next best thing. But at what point are we asking too much of our tester’s imaginations or cramming too much into one session?
How have you structured your questions in the past? Is this something you have noticed? What could we do to find out if this is true or not?