From idea to success through customer validation — a big fat lie
Newer innovation theories like The Lean Startup, Lean UX, Four Steps to Epiphany etc. focus on iteratively working your way from idea to successful product through customer validation. Typically when you test a product you focus on four areas:
- Utility (does it work?)
- Usability (is it useful? Do users understand it?)
- Desirability (is it valuable? Is it worth the money?)
- Aesthetics (is it beautiful?)
This article focuses on problems in relation to testing desirability (3) or the more fundamental and conserving issues: will somebody eventually use or buy my product? Something the above mentioned theories try to answer through scientific research methods.
However, despite great intentions, implementing the theories often end up doing more harm than good, slowing down product teams, their decision making and the value they add.
Many practitioners believe that moving closer to perfect information is a way to reduce risk in a decision making process and confuse the application of research methods. They think they can apply research methods on prototypes to answer questions about the desirability of a future product, in order to make a decision about whether to build the product or not. Reality is, that is not how it works.
This article highlights what you have to be aware of, to avoid the pitfall following the theories when developing products.
Why the models get misunderstood — the application paradox of research
The classical waterfall product development model was heavily adopted in the 90’s and 00’s in line with get-big-fast strategies. It is convenient, as it is a linear process, where you get an idea (the more genuine the better), build it, create hype around it and launch it. However, most product launches fail and most of them because there is no market need.
As a solution to that newer theories, coined by Steve Blank and popularised by Eric Ries in The Lean Startup, focus on bringing in the customer early in the process, and validating the market need, before building the actual product.
Even though it sounds like it makes perfect sense, again and again we experience a problem with the way these theories are being applied by most of the clients we work with:
Practitioners often misunderstand the application of scientific methods and believe they can get answers about future willingness to use or buy a non existing product, through e.g. a prototype by applying research.
It makes sense, why the theories have been adopted by corporate institutions. It is common knowledge that obtaining perfect information will reduce risk in decision making. That is also why, classical business tools, like discounted cash flow, net present value and scenario planning are popular within corporate institutions. But, when it comes to innovation and building something new you have to live with some unavoidable uncertainty — you will never know with certainty how your product will be used unless you build it.
The problem arises when the theories are used in the pursuit for more information to reduce risk, e.g. to research if there is a market need for a picture sharing service, by looking for underserved needs or frictions with existing services. When information is increased and it is decided to build the product, many believe that you will get certain answers if you apply the right research methods, forgetting that the context the research is applied on is key and that it constrains the answers you can get, especially on desirability.
What this misunderstanding often leads to is, that you build a prototype (probably believing it is an MVP), which is then tested with users to get an understanding of the desirability for the prototype; i.e. if users will use it, buy it or maybe share it with their friends and family. Users might even be asked to rate from 1–10 how likely they are to use and buy it. Then it is believe that this will give a better understanding of “how likely the product is to succeed”. But the truth is that this is good data to present to management to get buy in and approval to continue, however, it is not telling you as much as you want it to about the future of your product.
We call this belief in the quality of the answers you can get from research regardless of the reality in which the research is applied the application paradox of research.
The answers, and the quality of the answers that you get from research will rely on the context in which the research is applied. You cannot user test a prototype and obtain information about if the final product will be a success or not if built.
So, what should you do to avoid it?
In order to avoid this pitfall we suggest becoming familiar with how the reality to which research is applied determines the validity of the answers you can get.
Essentially, research allows you to observe what people do: their behaviour. Or talk to them or survey them about why they behave as they do, and what makes them behave like that, their attitude.
However, the context in which the methods are applied will lead to different results. What people do and why they do as they do can only be answered based on the reality they are in. This means, that if you test a prototype, you can get answers on utility, usability and aesthetics, on a prototype level. You cannot answer whether people are going to use, buy and or share your future product.
A prototype is a great tool to get a better understanding about your product and how it is intended to work and if people understand it. But it can never work as a validation in itself, no matter how many users you get to look at it.
A pretotype or fake door, e.g. the famous video example from Dropbox is a great tool to get a better understanding of how the product idea resonates in the market. It gives you an idea about the potential demand, but not a complete truth about the actual usage of your product.
Only a complete product allows you to answer how many people actually buy, use, share etc. your product.
Prototypes and pretotypes will give you indications, but not the same truth as a product will. To exemplify, Google would not have been able to get the same data about Google Glasses (e.g. people being assaulted for wearing them), if they had built a prototype and asked people how they would feel about someone wearing it and had not launched the product. Google launched their glasses, and only by doing that they got the knowledge about the bashing, which they can now use to inform their next decisions when it comes to innovation and developing products.
Summing up, you should of course bring in customers in the product development process. Be humble and acknowledge that you are not the customer nor inside their head. However, it is important to be aware of the pitfalls of the application of research when validating with customers, and how the context dictates what you can and cannot validate and the validity of your answers. You can only validate whether your customer actually buy and use your product by building it. So beware of the general misunderstanding of these new and often effective theories that way too often lead to a false sense of security and ends up being window dressing used to convince stakeholders and decision makers rather than being used to inform the product to make it valuable to the humans that is actually going to use it.
Increasing the fidelity to a level where you can get real answers on desirability, e.g. by applying the MVP mindset, is the next natural, but difficult step in the product development process and will be covered in another article.
Note: please feel free to comment and get back if any of the above struck your interest.
This article is part of Flux Newsletter by Hello Great Works. Sign up below.
