Why you should get out of your user test lab
Sometimes it’s so far off from real life it will not provide you with the insights you need.
The best practice police keeping you in line
As a UX designer you encounter a lot of “rules” or principles during your years in school on best practice and how to be a good designer. These also follow you in your work life. Stepping away from the rules can be uncomfortable, and colleagues may point out that you are not doing things in the right way. But the thing is, you are not a bad designer if you don’t follow the rules or best practices blindly, it’s rather the opposite. To me, best practice is the “safe zone”, which states what generally works or have worked before. It does not mean that it needs to be followed at all times, or that it will always be the best solution for your context and users. As long as you have thought through why you are changing things up, you should be good, or at least able to learn something. Usability testing is one area that can benefit from bending the rules.
“Keep (your test users) calm — and test away”
We learn that you should create a calm and stress-free environment for the user test. You keep your co-workers hidden away to observe, so no pressure is applied to the person participating in your test. You tell them they have all the time in the world, and keep them away from all possible noise and distractions. We spend time trying to create as “realistic” test scenarios as we can (which can be really hard), while not always thinking through if the test environment supports those real world scenarios.
Testing lab vs. real life
Say that you are developing a new heart starter or a first aid app, and you may think it’s a good idea to test it before you release it to any customers. So you bring some users in to your test lab, and you present them with the product and explain to them that their friend just had a heart attack and ask them what they would do. The test users manage to understand how to use it just fine, and your co-workers in the observation room give you a thumbs up. So you push it to market. Later it turns out that navigating through your service with a pulse of 200 is nothing like your feng shui test lab, and the consequence is sky high.
The same goes with (maybe) less critical services like Google maps. A huge portion of their users might be driving a car in a new country, or running late to a job interview while trying to navigate. Also, not a very calm and stress free environment. So why do we always emphasize that the user tests should be carried out in these fake environments of candy and peace?
Apply your design skills to your user tests
I’m not saying you should say a final goodbye to your user test lab, because sometimes that is the right place to be, and the quickest way to test something fast. (One of my favorite advantages of a test lab is the ability for the whole team to observe and understand your users struggles.) Just stop the autopilot next time and start to think what would be the best way to find the answers you need and why. And if the user test lab or test scenarios gets too far from the users reality, move out of your testing lab and closer to the jungle.
Right now at Norrøna we are working on a new checkout solution for our webshop. As you may have guessed, we are moving out of the test lab to understand how the new solution performs under realistic circumstances, before we deploy it to all our customers. This is a scenario where they are actually spending their own money, and the stakes are higher than in a fake shopping scenario we could have conducted in a user test lab. We will test in a less pristine test environment by visiting one of our physical stores, asking our customers to order the desired product on their mobile, instead of buying it in the store. The best part about this — we don’t have to put down a considerable amount of time figuring out realistic and good user scenarios to present to our potential customers — because they are already in it.
Here’s a list of 3 things to consider before deciding if your test lab is the right place to be:
1) First things first: Do you even need a usability test?
This is a question where you need to consider 2 things:
A) What are you trying to find out?
A usability test you would typically run in a test lab explores the usability of a solution — meaning you will not find the answer to whether this is an attractive service for your users or a profitable service for your business. If you are building a new product or service, a usability test will be a waste of time before you have validated your hypothesis if this is the right solution. However, if you are testing how to make an existing product right, a usability test could be a good start to find out how your customers are interacting with your product, and how the current design is working. Bottom line: You are not likely to find the true answer to whether your service will succeed and generate money for your business by running a usability test.
B) What are the risks of going to production without a user test?
I’ve experienced and been part of spending multiple hours of testing things to death before going to production — because we were too afraid of making mistakes. Good news is that when we are designing digital products they do not need to be perfect first time around, because we can do what we always talk about: Fast iterations! Do a small risk analysis of the possible business consequences of potential errors in your new design, and if the stakes aren’t too high, push it to production and measure the data and feedback from your users there. Bottom line: Deliver value as fast as you can, and get feedback from real users with real needs whenever possible.
2) Is the user environment similar to your test lab environment?
In which types of environment will your product/service be used, and is it easy for you to create a similar environment within your lab? If yes — good, if no — consider if there is an easy way to test elsewhere.
3) Are you able to easily recruit test persons that will be good representatives for your test scenarios?
It’s important that your test users easily can relate to the scenarios you will present in the user test. If the scenarios are too far from the test users real life situation, it will be harder for them to think and act like your real life users, and chances are you will miss out on important findings. Ask yourself if your product/service require a personal investment from the user (i.e. spending money, or making a big life decision) or a certain kind of knowledge, and if you are able to easily recruit test participants currently in this scenario in real life. If yes, high five to your recruiters, and if no you should find out where you can find people in this situation and test where the real needs and insecurities are represented.
Thanks for reading! 🤓
