What redefined UX evaluation could be?
We are living in the times of User Experience (UX). It is what sells your product now. It is what makes you competitive.
In the 20th century, with the coming of GUIs in software development, the focus switched from utility to usability — not only that something can be done with the product, but that users are able to find the functions and use them efficiently.
Today, this focus switches progressively to the overall experience with the product. Mind you, we are speaking here of any type of product — digital, like an app, and physical, like a car or a coffee machine. In principle, this means that the success of the product on the market is more than the sum of subcomponents like functionalities, usability, colors, customer relation management, publicity, etc.
Now, if it is the overall UX that is the driving success force, the question is how do we make sure that it is good before the product is released?
To be honest this is pretty inefficient today. It is a lot of fun, but each testing method available at the moment is time and resources consuming. And what’s worse, the most accessible ones (more on what that means later) are limited to what the users can self-report the testers. Sometimes the behavioral tests are done, but it is difficult to interpret the data, since we are missing access to the intention of the user.
Evolve or revolt?
So, what do we do now? Do we try to improve the existing methods or look for completely new ways that avoid limitations of the old?
Looking at modern technologies, we are ready to approach the needs of UX testing from new perspectives. Here are key concepts that are needed to revolutionize UX evaluation:
- ecological validity
- remote testing
- Data-driven insights
- Increase accessibility
- intention recognition
Ecological Validity
Simply said, ecological validity indicates that any test or measurement should be done in context resembling as closely as the possible real-life situation in which the product is going to be used. Already today the Usability Labs — that are the exact opposite of this principle — are becoming less and less popular. But some situation simply cannot be reproduced in real-world, e.g. when they pose danger to the user, like car accidents.
The other reason why ecological validity is tricky to maximize in tests is internal validity of the test. To definitively interpret the data, the tester needs to have control over variables — and in real life, we usually have a lot of things we cannot control.
Remote Testing
his one is straightforward. In many cases the product needs to be tested with the specific target group — e.g. students, before the first job, 25–27 years old, blond hair, height: 150–160 cm. Now, from my own experience, I can tell you that finding any users for testing the product is a tricky job. Finding specific ones — multiple times harder.
This becomes easier if the participant doesn’t need to come to a specific physical location. That is simply because then I can invite them from anywhere around the world.
Data-Driven Insights
Behavioral measurements, beyond video and observations, are rarely done — usually only in big projects. They are facing many challenges, causing it to be expensive, highly complex, and limited to physical location.
But, as difficult as they are today, they do provide necessary answers to evaluate UX of the product in great depth. The result is a highly optimized product for its target groups and therefore higher success chance on the market.
Increased Accessibility
The end goal is to make UX evaluation as accessible as possible. The more people are able to perform the assessments, the more often they are going to be done. Quite simple.
There are three important hurdles to solve so that accessibility can be maximized:
- results for dummies — this is not meant to say that people who can’t understand UX testing results are dummies. The UX testing data are notoriously vailed and multilayered. In many cases, they can’t be even statistically analyzed, which makes them even more complex. To make the evaluation more appealing — from team members to the bosses — the data and results need to be understandable by everyone
- tests for dummies — again, not meant as the offense, but as the warning. Many methods, including simple interviews, require a special set of soft skills to collect reliable data. And as usual, the more expertise is required, the fewer people can actually do the tests, the lest often they are done.
- reliable heuristic tests — I haven’t offended anyone with this one, have I? The experts in the field of UX testing know, that the heuristic tests provide rather unreliable data. Sometimes teams are lucky, and the designers are representing the target group. But in most of the cases, the designer is going to see the problem and solution from a vastly different perspective and approach it with a different skill set than the target user. Which means that the real users are needed for every test, which means that the tests are done less often.
Intention Recognition
Last but not least, there is the challenge of interpreting the behavioral data from the perspective of what the user wanted to achieve when performing a certain action. There are many advantages of having insights into intention, for example, to measure if the user achieved what (s)he planned, or that the observed result was just an accident.
Considering how complex this topic is, the field of intention recognition use cases is an unexplored area full of potential.
Why now?
Well, the answer is basically: the 2nd wave of virtual reality. Or what I like to call it the “Post-Oculus” era. For years now, a lot of companies have been working on things like full-body tracking, head-mounted displays, immersive displays, hand tracking, etc. However, the VR was for the most part inaccessible outside of R&D and corporations (again, mostly Automotive and Aerospace). The systems required a lot of technical expertise to even start, let alone use on regular basis.
But then, in 2012, came along Oculus Kickstarter and Palmer Lucky, with a dream to enable the VR experience to the masses. The focus switched from expensive and niche technology, to the mass production consumer devices. And long behold, many new startups and old giants discovered VR. The innovation in high-end hardware and the search for killer-app has begun again.
Today, there are many off-the-shelf high-end types of hardware and software available. From simple to highly complex body tracking, all in the 400$–4000$. From simple to highly complex finger tracking, all in 50$-5000$. Eye-tracking is about to be available by default in every headset (VRGineers’s XTAL, Vive Pro Eye, etc.). Tomorrow even more advanced equipment is coming. Full-body haptic feedback (e.g. Tesla Suit), high-precision force & haptic feedback for hands (e.g. HaptX), and a lot of other topics.
Follow the Spatial Debugger series to learn more behind every concept and use cases. Don’t hesitate to comment or contact me!
Email: sd@kizdeski.de
Medium: Krzysztof "RodionVR" Izdebski
Twitter: @RodionVR
