You might be wondering: what is Wizard of Oz prototyping?
Wizard of Oz (abbr. WoZ) prototyping is a technique used to test a concept/prototype that may not actually exist yet, but appears to be functional because someone is controlling it behind-the-scenes.
“Pay no attention to that man behind the curtain!”
My team used the WoZ to usability test a gesture based video control system with YouTube. Unless I’m secretly employed by Google and am unknowingly on their research team, YouTube doesn’t have functionality for gesture based controls (yet).
The Usability Evaluation
We were primarily practicing making our WoZ prototype believable, but our goals for the usability evaluation were to test the gestures we came up with.
We wanted to:
- test learnability of each gesture
- observe how our user utilized each gesture
- see which gestures were preferred and frequency of usage
- evaluate whether gesture did what user expected
The Prototype
Our prototype was a “hacked” Wii sensor and an Arduino hooked up to my laptop. The Wii sensor and Arduino were used solely for aesthetics to make the prototype look believable, whereas I was controlling the YouTube video on my laptop using a wireless keyboard.
The Wizardry
During the actual test, we had a facilitator (guiding the user), documentarian (taking photos & videos), note-taker (noting observations), and wizard (controlling the system).
Usually the wizard is in another room or hidden, but for this test, I posed as a note-taker, placing the wireless keyboard over an open laptop, while our facilitator drew attention away from me.
Participant Feedback
From the beginning of our test, we could tell our participant believed our system worked. She was surprised that her gestures were causing a reaction in the video.
During and after our tasks scenarios, our participant shared some valuable comments and asked a few questions about the gestures.
Feedback for improvement
- she was unsure whether the sensor was going to pick up her gesture (lack of extra feedback from the system)
- it was unclear how defined or how big the gesture had to be
- was not clear whether size of gesture affected the result (i.e. does a larger gesture for speed up affect how much the video speeds up by?)
- a few gestures seemed unnatural or random
Positive feedback about the prototype
- liked the concept and would use it at home
- prototype worked well and was cool!
In-class Critique
When we presented our video in class, we generally received positive feedback on the test and video. People said:
- the wizardry seemed convincing
- our task/test structure was simple and easy to follow
- the video was straightforward and well labeled
- it was good to provide user with feedback that the system was working
Reflection
Overall, I think our prototype was convincing and it was really fun to use this technique! Although we didn’t write a comprehensive usability evaluation, the point of this exercise was to test the prototyping technique than the gesture system itself.
During our test, the system worked a little too well because all of her gestures were recognized, making it feel like the prototype was too refined/too good to be true. For next time, we would try to make the prototype even more convincing and generate more research findings by not making the system perfect. This would also show us what the user would do in case the gesture wasn’t recognized.
