This I believe:
Successful products succeed uniquely but failed products all fail for the same reasons.
In this post I’ll describe the 5 most common product fails I’ve encountered and what I’ve learned to do when I see them coming.
1. The Top Down Fail
When the Hollywood film is made about the Volkswagen emissions cheating scandal my bet is the story it will tell is one where the senior executives of the company made promises to investors that engineering could not keep. This is a classic version of the top down fail.
This root of this failure mode is a good idea, actually, a great idea. In fact, you cannot have a top down failure without first having an idea that is so brilliant, so simple, and so obviously true that it can’t be wrong — except that it is!
In Volkswagen’s case the idea was Clean Diesel. Exciting financial projections for this aspirational brand were shared with investors and promises were made. Those financial projections became someone’s responsibility to make tangible, first, as a marketing campaign and second as a product. Except, the product part had to be a lie because it had turned out that Clean Diesel was not actually a thing — diesel passenger cars with low emissions and zippy performance could not be built.
So, given this fail starts with powerful people, how do you avoid it? OK, first, keep it from going from Top Down fail to Criminal Top Down fail by admitting it before you find yourself lying to the regulators! But, more practically, you can often nip this fail early if you make it a habit of looking not only at ideas, but at where they come from.
Avoid top down fails by looking at where the idea comes from.
Ideas that come from the bottom — meaning from people without control over budgets — have to go through an extensive vetting process to get funded. It doesn’t matter what the institution, a universal truth is that an idea starting at the bottom must prove its value before it gets funded. And the only way this happens is that idea meets skepticism, gets tested, reacts to feedback, gets knocked down and gets back up so that when it finally gets in front of the powerful people with budgets there is a good chance it actually is a good idea. It also comes along with a champion, a person who believes in the idea so strongly she has been willing to get through and past all the resistance.
On the other hand, an idea that begins life already in the mind of a person who controls a budget faces two risks. The first is that it has a much shorter path to being funded. And on that short path it doesn’t get poked, prodded and tested in the same kind of way as a bottom up idea. The second risk is that when this idea does get budget, because it comes from someone important and busy, it will be given to someone else to execute, most likely someone without a true passion for it.
If you come upon this fail in progress, before it is truly terminal, but well after you can just just shut it down without a ton of fallout, then the only fix is research. Take the top down idea, expose it to expert scrutiny, get real user feedback, and iterate on it until you find a version of it that works top down and bottom up.
2. The “Feature Parity Will Be Easy” Fail
Speaking of bottom up, consider NONMEM. This analysis software you’ve never heard of was created by my father, Lewis B. Sheiner, and his long time collaborator Stuart Beal, over 30 years of working together at the University of California, San Francisco. NONMEM is about as bottom up as it gets. It is a seminal piece of technology in the history of machine learning; a federally-funded project that enabled data scientists to use computers to model and simulate statistical predictions long before there was S or R or Python Pandas. Or even PCs for that matter.
My dad was the theory and ideas guy; Stuart, the mathematician and engineer. When a venture capitalist & software entrepreneur came along wanting a license to commercialize the technology my dad said yes but Stuart wouldn’t sign. Some years later I ended up working for the start up founded by that same entrepreneur & VC pair that set out to write a statistical engine to compete with NONMEM. Our engineers were certain that with their professional techniques and modern languages they could quickly and easily exceed the capabilities of ancient, command-line NONMEM written in Fortran by an self-taught academic programmer.
Except they couldn’t. Or at least not in anywhere near the amount of time they had predicted. It turned out that in 30 years of tinkering Stuart had produced a rich set of bug-free features that were optimized from every possible performance angle. The NONMEM algorithm was far faster, more flexible and more consistent than anything our engineers could produce. The engineers had been caught by the Feature Parity Will Be Easy fail.
The best way to recognize this fail is to notice a schedule looks kind of like a Big Bang and smells a bit like hubris. In other words, if you encounter a product plan with the idea that before a certain date there will be no functionally useful thing and then immediately after that date there will suddenly be a very useful thing, as good as or even better than the competitor, you should get suspicious of being headed for a Feature Parity Will Be Easy fail.
If your schedule looks like a Big Bang, beware the Feature Parity Will Be Easy fail
There is no easy fix for this fail. The best way to handle the bad news, though, is all at once. As soon as you realize schedule expectations will be missed, grab the executives and give them a worst case estimate. Be honest with yourself and others about the nature of the gap, the size of the miss, and the time needed to close it. This will not be a fun meeting, but the outcome of keeping quiet will be worse. And if you can’t get support for the new plan, you are dealing with magical thinkers and you need to leave.
3. The “Customers Will Rapidly Adopt Changes” Fail
Everybody wants bright, shiny, better and new. Right? This is what we tell ourselves, if we think about the issue at all, when we consider why our users will accept breaking changes that we make in our products.
As the exception that proves the rule, consider Apple. Apple frequently releases hardware with data ports that don’t work with existing accessories. People complain, Apple ignores the complaints, customers are pissed but remain loyal and lots of adapters get sold. Wash-Dry-Repeat.
The fail here is thinking that you can be like Apple. First of all, Apple has done this successfully a bunch of times and you have not. Second, when your customers — the ones paying you, the ones who could punish your brand on social media in a click or two — push back on the changes, are you really going to stand fast?
Sadly, most companies don’t even think it through this far. Instead, after they have invested in building and releasing a new, better but backwards incompatible product they are shocked to discover that customers have no intention of switching. Why would they? They already paid for one experience, they have built habits and business processes around it and have no incentive to disrupt all that just because, after a ton of hassle switching, they might have a better experience.
So customers do the rational thing and do not adopt the new stuff and most companies can’t or won’t make them. The reasons might be contractual, or fear of brand erosion, or some technical or physical constraint. It doesn’t matter why, but the bottom line is that if you don’t force your customers to adopt product changes, most likely they won’t.
If you don’t force your customers to adopt product changes, most likely they won’t.
Instead you become responsible for maintaining two systems, the old one with your all your old, loyal customers and the new one with your new customers. Over time the two systems become more and more incompatible and your customer base less and less aligned. The cost of this redundancy works like a tax on the resources you have available to innovate. At some point this situation becomes seriously dangerous to your future.
There is only one possible fix for this fail and it is not pretty: pay your customers to switch. The implementation of this can take several forms, from investing in building great experiences that automate the transition to giving your customers free consulting services to adopt the changes manually. Or, absolutely worst case, by rolling the changes all the way back. But bottom line, it happens on your dime.
4. The “Integration Is Not A Feature” Fail
Imagine someone tells you this:
“Hey, listen, I have this great product that you will love. You can try it for free. If you try it, you will love it. After an hour of set up you will be able to see for yourself how much you love it.”
Would you try that product? I don’t mean hypothetically, I mean right now, this minute, would you actually put your day on hold for an hour to set up something that might be great but also might be worthless? Most likely, no, you won’t.
If your adoption strategy sounds anything like this, you are part of an Integration Is Not a Feature fail. The challenge here is that these fails are hard to see unless you take your eyes off the sexy cool part of your product experience, and look at the boring, tedious corners no one wants to think about.
For example, I once worked at a streaming analytics start up. The pitch was “connect your data to our system, and we deliver to you useful (and sexy!) real time charts and dashboards for monitoring your business.” This really was true, we really could do that, and yet when we offered free trials, no one signed up. My UX team was sure our set up experience was the adoption blocker: it involved many complicated steps, each one easy to mess up and hard to debug. However, I couldn’t get anyone with budget to take the issue seriously, because, hey, our charts were awesome!
So I used Craigslist to find 3 people who matched our target demographic and paid them to try our free demo. I gave them written instructions and asked them to complete an end to end set up so that they could send data from their system and see it in ours on a test dashboard. Two people got it done in about 90 minutes; the third gave up in frustration well before that. Only after I showed these results to our CEO did I get funding for a project to productize the set up experience.
The fail here is that when thinking about a new product that depends on an integration to deliver value (and really, don’t they all?), product strategy tends to focus exclusively on the new features the product offers without also allocating budget to treat integration itself as a feature, possibly the most important one.
Integration is itself a product feature, possibly the most important one.
5. The “A Technology is Not a Product” Fail
The high level picture of this fail is where the thing you are marketing as a product — a solution for a use case — is actually not a solution but a technology that can be used to build that solution.
Let’s try an analogy. Imagine that what you wanted was kitchen cabinets. And I heard about your need and tried to sell you a table saw on the basis that you could use my table saw to build your cabinets. And, in fact, it was a great buy because you could use it to build not just your cabinets, but almost anything!
You would probably think, OK, fine, but the thing is, I don’t want to build anything! I just want some cabinets!
That’s what the “A Technology is Not A Product” fail feels like. It occurs when people selling a thing see customization as a feature but other people just see it as a lot of work.
In my experience this fail, fatal as it is, originates from a pure, honest, even sort of altruistic place. What happens is that some technologists get together and build a cool technology. And that technology is cool partly because it does some innovative thing really well and partly because a skilled practitioner, given a reasonable amount of time, can use it to solve all kinds of problems. Just like a table saw.
However, the fail is that the technologists have forgotten — or, let’s face it, probably never really understood— that most people are not like them and don’t want to build things, they want things.
There are two fixes here. The first one involves convincing the technologists that the only way to keep their beautiful technology from going extinct is to use it to build a real, bona fide, appropriately targeted and immediately useful product, and then sell that. The challenge here is that even if you are successful at getting this idea across to the techno-execs, and even if they believe it, they may not have enough running room left in their funding model to execute it.
Which brings us to the other “fix,” which, short term, is more likely to succeed but is a lot more ethically compromised. In this fix what you do is create a credible marketing narrative about the technology solving a real use case and you support that story with a working demo — maybe you even have an actual customer testimonial too. The use case and the demo (and the testimonial) are all real and work to convince people that “with just a little bit of customization” they too could have a slick solution to their use case.
In the best case, this approach brings your technology to the attention of a builder, a person who actually does like to build things, who can buy and use it to solve a very specific use case. However, more frequently what happens is the company, desperate to recoup investment, targets people and organization who can be wowed by a cool demo and will sign a contract without really understanding the amount of customization work they will have to do to your “product” before it is actually useful to them.
When you do this you can tell yourself caveat emptor and all that but, speaking from personal experience here, you know what you’ve done and it does not feel good.
As a final note on this fail, if you are thinking “hey, fail #4 and #5 seem pretty similar” give yourself 10 points for paying attention. The set up problem is similar in both cases. The difference is, #4 is about a situation that can be fixed by investing in the set up experience whereas #5 is about the situation where set up challenges are just the most obvious symptom of the underlying mismatch between delivering a product versus delivering a technology. Returning to our favorite analogy, if what you want is cabinets, my table saw won’t help you regardless if I just drop it off in front of your house in a crate or if I also send along a wonderfully polite technician to unpack and set it up for you. Either way, it’s not cabinets.
Where To Go From Here
Hopefully after reading this far you’ve got some new useful perspectives you can use to evaluate the business context for whatever project you are currently putting your passion in to.
If you like the idea of thinking about the decisions surrounding the work you do in terms of fundamental patterns or archetypes, then you are a system thinker. Read Thinking In Systems by Donella Meadows to learn more about that.