Usability Testing for XR products
Usability testing is one of the important aspects of user experience research that helps to understand the product's ease of use and functionality. In this, you can observe how the targeted users interact with your product and can complete the tasks you want them to do via your product, hence bringing value.
With XR products, usability testing becomes more important because of the set of challenges XR brings like device complexity, different interactions and new technology.
What is Usability Testing?
It is a method that aims to evaluate how easy the users find it to use your product and is able to achieve a task. It involves multiple steps like identifying target users, observing the usage physically or remotely and collecting their feedback.
Why is usability testing important and different for XR products?
Extended reality (XR) product refers to Virtual Reality(VR), Augmented Reality(AR) or Mixed Reality(MR)products. This can be hardware products like headsets, glasses, HMDs or software products like games, websites, tools etc.
For XR products, it becomes more important because of challenges like:
- New and unfamiliar technology
- Need for specialised devices like glasses, HMDs etc
- A different way of interaction in a 3D virtual environment
- Possibilities of cyber-sickness
- Setting up device and environment to experience
When to perform usability testing?
It is an iterative and recurring process and is good to perform in different stages of the product. It can be:
- Early development- It is good to check the usability at the concept stage before writing the line of code. This can be through paper prototypes or basic user flows.
- Prototyping- Prototypes are good ways for users to visualise and interact with the product. Any relevant feedback here can be super helpful before developing the beta product.
- Alpha testing- This testing is for only internal users and is intended to identify any usability issues that users face to reach their goals.
- Beta testing- This testing is usually done with external groups to identify any remaining usability issues that users face in the overall user experience.
- After development- As mentioned previously, usability testing is a recurring process. Hence even in the later stages, it is always good to have this activity with your target group. It can be for a different hypothesis to test or a change in user behaviour or any update in the product or a change in the goal that the user is trying to achieve.
Practices for the usability testing
Practices before conducting the test:
- Target users- Identify the target users and make sure the test participants are representing your target group.
- Build hypothesis and script- Create a hypothesis on what needs to be tested, and improved and what the user is trying to achieve via the product and the value it provides. If there is any existing data, analyse it to create the hypothesis. Build the script, action tasks, and story to narrate.
- Setting up the test environment- XR products need a physical environment that is comfortable and conducive. Test that it is safe, has the least distractions, no obstacles, controlled light and enough space.
- Setting up the devices- XR products can be complicated. Therefore it is important to set up and check VR headset tracking, AR glasses calibration, configuring MR systems, fitment or even maintaining the hygiene of device usage. Also test the device in different lighting conditions, in different physical spaces, and with different types of input devices.
Practices during the test:
- Building scenario- Create a scenario and explain that to the user. This helps users get the context, imagine the scenario and understand what they need to do during the test.
- Collecting mixed feedback- A combination of qualitative and quantitative data is good. It can involve both collecting feedback from users (qualitative data) and measuring objective performance metrics (quantitative data).
- Facilitation- XR product testing needs more facilitation from the interviewers than conventional usability tests. Be vigilant about verbal and non-verbal clues. Also, since you can’t experience what the user is seeing simultaneously, you will need to ask questions keeping in mind that you don’t ruin their natural flow of experience.
- Non-leading questions- Avoid leading questions like “Will you click this button to search the videos?”. Rather ask non-leading questions like “If you’re looking for videos, how will you find them?”
Steve Krug has mentioned numerous situations in “Things a Therapist Would Say” that can happen in usability testing.
Practices after the test:
- It’s not an exam- Make users comfortable and tell them it’s not their test but only of the product. Ask about their overall experience and if there is anything that was missing or if they want to share more. Any feedback can be helpful.
- Note-taking- A dedicated person can record the feedback. It can be through video recording, audio recording or pen and paper.
- Compile, analyse and report- Once feedback is collected, compile them, analyse and note the insights. Understand the user behaviour, how it fits into your hypothesis, and what are the problems to solve and prioritise them. Successful usability testing is not just about getting feedback but knowing the problems to be solved followed by immediate action.
Always focus on the most serious problems and not on the low-hanging fruit and things you’d like to fix.
Takeaways
XR being an emerging technology with high penetration, new products, multiple use cases and growth, the importance of usability testing becomes more important. Though usability testing has its conventional ways, with XR there are significant differences that are needed as mentioned in this article. With such iterative usability testing sessions one can save time, energy, and cost and reach a better market fit which is critical for the XR industry.
