Why User Research will save you from doom [possibly]
I recently read a couple of blog post that — for different reasons — made me chuckle and sigh at the same time, one titled ‘User Research is Overrated’, one titled ‘You’re wasting your money on research’.
Let’s say it: I am biased. I’ve spent all my life in research: scientific research, R&D, market research, and now UX research. I was therefore curious to understand why my job should be sort of redundant. I learned insightful things such as “Up-front User Research is [...] a way to avoid making hard decisions.” or “the useful data came from the first user tests, not the research.” And also “the way it [research] is currently consumed means that it doesn’t always improve decisions”; “our research is reaching the wrong people in the wrong format at the wrong time”.
Since “every truth has two sides” (Aesop, 620–564 BC) I feel free to share mine. I will also try to support it with facts, as I tend to leave unsubstantiated opinions to Facebook posts and consider them a good reading on the toilet seat.
Disclaimer: This started as a short blog post, but turned quickly into a very long post. I decided (was strongly encouraged to) split it into three parts; in a nutshell, I am trying to convince you that skipping user research can actually be detrimental and lead to failure. If I succeed, and you keep on reading, I’ll warn you against a number of pitfalls research can present, because in the end bad research is equivalent to no research, plus you have wasted money. Finally, I’ll touch on how to convey your message, to prevent that “all those moments will be lost in time, like [coughs] tears in rain” (Roy Batty, 2016–2019).
Now you decide whether to keep on reading or not.
PART 1: SKIPPING RESEARCH IS LIKE SKIPPING BREAKFAST — you can do it, but in the long run it does not take you far
STORIES OF FAILURE
My story starts in 1992, when Crystal Pepsi was released. Do you remember it? Who does? If you do it’s because they briefly tried to resuscitate it recently. Anyway, the idea was to dip into the market trend of clear colour beverages, in order to convey message of pure, healthier product. Are we talking about a pure soft drink? Yes, we are [#scoffing].
Had they conducted some research they might have learned that yes, their assumption that consumers were getting smarter about their health and welcoming more natural products was correct, but Crystal Pepsi was not the solution as not perceived as healthier, and it tasted just like Pepsi.
“People were saying we should stop and address some issues along the way, and they were right.” said David Novak, CEO of Yum! Brands and credited with introducing the Crystal Pepsi concept. “It would have been nice if I’d made sure the product tasted good. Once you have a great idea and you blow it, you don’t get a chance to resurrect it.”
Yeah, but that’s consumer goods. What does have to do with UX research? We are designers, we empathise with our users by default!
Research is research is research, mate! According to Oxford Dictionaries: British and World English it is the application of the Galilean scientific method of systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses. But I’ll come to that later.
Now, I know the word ‘scientific’ carries this stigma of obscure, unintelligible and cold, but hell no, it’s a mindset and a framework. I also have read posts about differences between user research and market research. Nah, the framework still holds. So I, a researcher, do not find in principle an overwhelming gap — in meaning or framework — between market research, social research and UX research. In the following I’ll refer to them interchangeably, whether you like it or not.
In a world so heavily affected by survivorship bias, with the consequence of looking at features that successful products have in common and assuming that they are the reason that makes them successful (a couple of nice articles on how it distorts reality can be found here and here), I am currently quite interested in failure (and reflecting upon my own).
So I’ve been pondering on why most products (or even startups if you want) miserably fail. The percentage varies according the source and category ranging from 40% , to 75% to 90% (also here) or even up to 95% (and here).
One — not the only one, but definitely one — of the causes of failure is skipping the research phase and basing the design and development on assumptions, rushing to the design and delivery phases without “getting out of the building” (as recommended by Steve Blank).
Anyway, let’s get more techie, let’s talk Kodak.
Since its bankruptcy in 2012 Kodak has been subject of case studies and articles, featuring the Wall Street Journal and Forbes to mention two. There were several factors contributing to its decline, one of them being they did not understand the shift in paradigm that the digital photography was causing. Despite they actually engineered a digital camera as early as 1975 (they filed the first patent on digital camera in 1978) they dismissed it for the very reason it was filmless. “That’s cute — but don’t tell anyone about it.” (Steve Sasson via The New York Times 5/2/2008). Some user research would have unveiled changes in the target market and user needs.
And what about the 3D TV?
Driven by the success of the movie Avatar — which was released both in traditional and 3-D formats — 3D was announced as the new black (and source of revenue). As in the previous examples, a lot factors contributed to its failure, but one is certainly the fact customers dislike having to wear glasses to see 3-D effects. Plus, nowadays at home a huge number of people use other devices when watching TV. How can you do that wearing 3D glasses?
On a personal note, I went twice only to see 3D movies: Harry Potter and the Deathly Hollows pt 2 and Gravity. As a short sighted person wearing glasses, having to wear another pair on top, was a discomforting experience to say the least, which distracted me and prevented me to fully enjoy the movie.
Guess what? Vizio, consumer electronics manufacturer leading the way in 3D smart TV admitted they did user research only after launching the product (mmmh, that reminds me of the user testing of the aforementioned blog post). “We can’t actually do it prior to the launch because consumers don’t have the product,” said John Hwang, senior product manager for Vizio’s HDTV category. “When we first launched 3-D, there was no consumer data. A lot of data was pulled from the movie experience, but movies are different.”
Had they done some research they might have learned that “buyers, no less than sellers, come into the market to solve problems.”( as Wharton marketing pioneer Wroe Alderson said in a 1954 lecture). Perhaps 3-D TV was the answer to a problem no buyer had.
Finally, I know if I do not show you an app you won’t be convinced, so here we go: Hailo, the app to ‘hail’ cabs from your phone.
Basically an Uber, but for regular taxis. There was a flawed assumption, on the audience, which they did not bother testing with a little bit of research: taxi drivers need help finding fares. Turned out, that was not actually the case in NYC; Hailo just wasn’t necessary. Moreover at that time not many taxi drivers were carrying smartphones.
User research would have helped understanding the audience, and ensure that the product is a solution to a real problem.
And what about the plethora of entrepreneurs and marketing managers seeking help of consulting firms for their “revolutionary” products? Usually, when they are asked about the research supporting their claims, they reply “We haven’t done the research yet, but we know anecdotally that it works and is totally safe.”
One of the main reasons why most products fail is that they do not solve a problem, or they brilliantly solve a non-existing one. Companies may have a great intuition about a feature or a concept, but they do not put the time (and money) to define the target market, understand the users, their needs and frustrations, grasp the essence of the problem they would like to solve. And that’s what you do with research.
Ok, have I convinced you to do some research to understand the user and the problem before jumping to designing the solution?
If yes, keep on reading to find out how to cross the research minefield, if no, I’d love to read your (fact based) point of view.
