VR Usability. Part 3
Why and how to test VR apps
Article by Olga Ivanova, contact via olga@vroxygen.com
VR, AR (XR) User Testing and Usability Studies. Contact Us:
VR Oxygen Home
Testing XR (VR, AR) Apps | Sharing the Findings of VR Design | Learn more!
vroxygen.com
Key Findings, Analysis, Hypotheses
And lastly, I’d like to talk about analysis and evaluation. After the tests are done it’s time to study the findings and make some hypotheses, you can create affinity maps or diagrams to help you to sort out the discoveries. So you can see if patterns appear.
For example, who and why liked or disliked something. Or connect why they disliked based on what you’ve learnt about these people.
Analyzing
It’s always useful to evaluate app performance (Load Time, Frame Rate and if there are any drops, lags, bugs). Of course, comfort, 3D Art, UI and navigation, gameplay and game mechanics as well.
When doing tests you can ask people to rate the app as well, or rate some features of it.
(Later you can find out The Net Promoter Score — it’s an index ranging from -100 to 100 that measures the willingness of customers to recommend a company’s products or services to others. It is used to estimate the customer’s overall satisfaction with a company’s product or service and the customer’s loyalty to the brand.)
You can rate an app from 1–5 or 1–10 scale depending on what you rate and its complexity.
The overall app rating should be from 1–5 scale as it usually is (like in App Store, Google Play, Oculus Store, Steam etc.).
Finding both, what people like or dislike is beneficial, because it helps to iterate and move in the right direction.
Key Findings
Let’s check some of the findings now. These are from VR Oxygen’s research a while ago when the goal was to find out what people do when placed in VR, specifically, first-time users. How they use certain features, their feelings and thoughts.
People don’t turn around/ look around for cues.
At several expos VR Oxygen demonstrated Cardboard VR app which was selected as one of the favorites at that time, because of its beautiful graphics. Some of the discoveries were surprising at that time.
Even when people were told that it’s a 360 experience where they can move around and look around, they wouldn’t. They would stay still and wait, some would turn head very slightly to the sides, carefully. And it was because they were not used to it. New Users also didn’t look down, even a little, but looked in front of them.
You can see in the video below that the player got bored waiting for something to happen and put the Cardboard down. They didn’t understand they should look on birds and didn’t notice that the pointer was diluting or didn’t associate it with an action they could take.
In this situation the Hypotheses are pretty obvious:
Creating clear visual and audio cues at the very beginning and positioning the UI and important elements in front of the viewer will explain players they can interact with the environment, will guide them.
If a scene is engaging , with clear directional cues, more likely people will turn around and explore the full 360.
Getting people to understand within the first seconds that they can move around will help to keep them engaged.
The next finding:
People assumed they could perform any gestures. People tried to manipulate all the objects.
Some people did that or something similar after encountering that items are interactive in VR, so they tried to manipulate all the objects that were present in the environment. The others assumed that if it’s VR they would see their arms there. They tried to grab things.
Based on these findings it’s easy to make Hypotheses that:
– All the objects should be interactive and useful, behaviours — easy to understand in order to support axpectations.
– Creating interactions based on real life affordances and people’s mental models will provide smoother lerning curve for New Users and may help to predict behaviors.
Tips
– Always keep in mind the context, map a User Journey to see end-to-end experience.
– Don’t rush or start the experience automatically, make the user control it. Agency is important.
– Provide users with a clear feedback on what’s happening, how they are doing
– Use omnidirectional sound in general at all times
– Recruit the right people for tests
– Consider ergonomics (Avoid interactions that can cause “Text Neck”)
(Make sure that labelling is clear for new users)
Usability testing is too important to be neglected. Start testing early, iterate and test again and so on until the best option is found.
Read Part 1 here
Read Part 2 here
Share your own story and user testing methods with us,
Contact for collaborations or just say Hi!
Please, contact via olga@vroxygen.com
VR, AR (XR) User Testing and Usability Studies. Contact Us:
VR Oxygen Home
Testing XR (VR, AR) Apps | Sharing the Findings of VR Design | Learn more!
vroxygen.com
Read more of my articles:
Check my Case Studies:
Visit vroxygen.com
Watch the Usability Study video:
TEST your VR app NOW:
