Doing voice research is very much different from usual forms of user research. And yes we’ve learned it the hard way. We’ve turned our experiences into some tips and tricks. Some of the them might seem obvious, but believe us, they’re not. Use them to your advantage :)
Pretest
- Take time to explain that the participant is going to test a VUI. Sounds logical, right? Well, it isn’t. Would you ever explain to a participant what a desktop computer is before running a user test? Probably not. Not everybody knows what a Voice Assistant is. Let alone how it works and what you can do with it. Explain this briefly and have the participants ask a few simple questions to the Assistant to get comfortable.
- Emphasize that it’s alright to make mistakes. Generally, people find it exciting to test VUI’s. Put participants at ease by saying that you invited them because of their limited experience and because you want to test the product now that it’s still new and flawed.
- Don’t ‘think-out-loud’! We always ask participants to think out loud because we want to understand what goes on in their head. VUI’s are an exception to the rule, for the simple reason that Voice Assistant are always listening.
- Uncover the participant’s experience, expectations and opinion on VUI’s before showing the test material. For an important part, this will explain the way they react to the VUI. People with more ‘voice experience’ know better how to give voice commandos for instance. Besides, understanding the participant’s opinion on VUI’s before the test, allows you to spot changes of attitude afterwards.
Observation
- Determine when you want to ask questions. Asking questions while observing is not a great idea, because the VUI is listening in and might think it’s fun to react :) Besides that, questions disturb the participant’s flow. Try to save up your questions and ask them afterwards.
- The setting of the test matters more than ever. When testing a VUI, the setting is particularly important. Why? Because the relevance of VUI’s heavily depend on context.
- We can’t see what goes on in the Voice Assistant’s ‘head’. Meaning you have to pay extra attention to understand what’s happening exactly and why certain things go wrong. For instance: does the Voice Assistant not understand the commando or does it not hear the commando properly?
That’s it for now. In the next update we’ll share some tips and tricks on the posttest and the reporting of voice insights. Stay tuned!
UPDATE: Read pt. 2 of this update right here.
Questions? feel free to reach out at hello@valsplat.nl! Do you want us to keep you updated through email? Sign up for our mailing right here.
