Stop validating with your users
I have been in this game for a looong time, and I have experienced first hand the emergence and maturation of the modern UX discipline as it applies to digital experiences.
We (the UX community) have fought hard to prove the value of UX — to product management, engineering and businesses executives. We have have convinced those groups that we can reduce the risk associated with projects if only we can spend time up front validating our designs and our assumptions and we tend to do that by putting paper sketches, wireframes and prototypes in front of real users to understand whether our solutions are meeting their needs.
Well I’m here to tell you that we need to STOP spending so much time VALIDATING with users!
OK, so before folks freak out and I am stripped of my official UX membership card, let me explain what Im talking about.
Victims of our own success
So, there is no question that our design discipline has come a long way in the past 20 years. In the past we often had to fight tooth and nail in order to be able to do a little bit of research, or testing, and now our design teams are becoming autonomous and have gained the freedom to do all the research and validation that need.
“with great power comes great responsibility…”
Now that many of us have the power to do what we want… we need to make sure we are doing the right things and not leaning too much on validation to the point of becoming counter productive.
The value of UX
Anne Hjortshoj — one of my very smart colleagues — always says… “UX is all about reducing risk, and revealing value.” We want to help our product teams ensure we are choosing the right problems to solve, and then make sure we are solving them in the right way. One way we set out to reduce risk is to validate our designs with users so that we figure things out “on paper” vs spending expensive development resources building it.
Now this all sounds great but there are a couple things I think we should keep in mind.
#1 — Risk vs Reward
When you have the freedom to validate anything you want, it can become really easy to just start validating everything! Sometimes it can even become a crutch where the designer simply stops making smart, justifiable decisions because they can always just “take it to the users.” The problem with this approach is that at the end of the day we need to serve the business as well as our users and it can be really easy to slow the overall progress of a team if every decision gets validated before building it.
One of the things my teams are starting to consider is the idea of risk vs reward. Whenever we feel the urge to validate with users, we first think about the amount of risk represented by the design at hand. How much will it hurt if we get this thing wrong? Is it hours, days or months of lost effort? How much risk can we tolerate? Asking these questions often helps understand whether we should just make a smart design decision and validate when its built, or do we spend the time to test it immediately.
#2 — Seek learning, not validation
I think that we need to shift the way we think about design validation. The idea of validation is often flawed because it can be about affirmation rather than what it really should be, which is to try and Invalidate our ideas.
Rather than thinking about it as validation, I believe it needs to be framed as learning. We should always have a question that needs to be answered, or a hypothesis that needs to be tested and then we can create an experiment that accomplishes those things.
#3 — Always be shipping
Our product teams are currently embracing the idea of “always shipping.” As a result, we are starting to look at our projects as having two phases.
Phase 1 is value discovery, which is all about identifying the actual needs of the user and figuring out how to satisfy them. The idea of shipping in this case is about rapid experimentation where we put utility into the hands of a customer (not a prototype) and measure whether that utility provides them value. Its important to note that value only exists if it can be measured, and so whenever we plan to build something, we also plan to measure it.
In value discovery we are not focussed on perfecting the experience but rather just getting the basics nailed down. These experiments — which are run with a small handful of users — reveal what provides the most value and it often points to what will be required for a minimum viable product (ie something we can push to all our users) and once we know that, then we can shift to the next phase.
Phase 2 is value optimization, which is all about revealing as much of the value as possible and this is where we focus on the experience. This is where we care very much about the usability of the solution. We want to ensure its discoverable and frictionless. This is also where we apply the level of polish associated with something that will go out to our entire user base. Measuring value remains important here and so we will ensure we define success metrics that can be measured at scale when deployed to all our customers.
The most important thing about the idea of shipping quickly and often is that it allows us to learn in the most effective and accurate way by putting real utility into customers hands that they can USE rather than simply pretend to use.
Aiming to ship utility every couple weeks during even the most complex project forces the team to focus on small slices that represent the highest priority capabilities and getting them right.
There are always dependencies
It should be noted that working this way requires that a few other things are in place within the organization:
- the PM and Eng teams are willing and able to work in a lean and agile way.
- You have access to users/customers that you can work with to iterate.
Wrapping it up
So, back to my initial statement about not validating with users, this is what I meant:
- Don’t use validation as a crutch. Always evaluate whether the risk vs reward makes the effort of validation worth while.
- Seek learning over validation. Don’t try and prove yourselves right… rather try to prove yourselves wrong!
- Consider the value of shipping over lots and lots of design iterations on “paper”
- focus first on value discovery, and then move to value optimization.
