The Fallacy of Data-Driven Product Design

In the era of big data, lots of companies build their products using data, so as Netflix and Amazon.

In 2013, both of them launched a political drama claimed to be built on top of data. Yet, these two data-savvy dramas received completely different results after the release:

Netflix’s House of Cards hits a big success which scores 9.0/10 on IMDb and is popular around the world; while Amazon’s Alpha House just reached an average score of 7.6.

Now, the question comes: Both of the shows are built on top of data, but why does Alpha House fail to succeed but House of Cards is a homerun?

To answer the above question, first, we need to understand the difference between their approaches on “data”.

It’s not about data-driven. It’s about data-informed.

Alpha House’s team relies merely on cold hard data to make decisions. They rolled out a competition of drama series and invited viewers to watch the sample show. They collected data about user behaviors (such as playback, pause, exits etc.) and collected data points on favorite elements in the shows. The results? A show with four political republican politicians — yes, it’s Alpha House.

On the other side, the team in Netflix also analyzed a lot of data. They found out the top topics, stories, casts, crews, elements in every show. Then, instead of putting all elements together like Amazon did, they combine their imagination to build a drama — House of Card. Netflix did not use data to conclude something. They look at data and make assumptions.

Netflix adopts the data-informed approach. Data-informed is more hypothesis-driven. It helps you to understand the situation, and allows you to verify your hypothesis — but it does not propose a solution itself.

It’s the data-driven concept actually drags Alpha House away from the victory.

You interpret the data, not driven by the data.

Data is the answer. Without knowing the question, the data itself is meaningless.

Many people, including the smartest decision makers at Amazon, place data in a wrong position. They use the data to decide the next step — “That’s what the data indicate! We should do that.” It’s undoubtedly a much easier decision since data has shown you the way. That’s why it’s a common approach for people to try to find a “Golden formula” and engineer the success. Take Alpha House as an example, we learned that being data-driven might not be useful in every aspect.

So, what should you do with the data?

Understand the users behind, not just the data itself.

“If I had asked people what they wanted, they would have said faster horses.” — Henry Ford.

The key to great product design is not just about data. It also about something behind it — customer empathy and your gut feeling.

Build up customer empathy
Customer empathy is providing you the ‘why‘ for the data you obtain. It can be built up via listening to your customers. You can achieve it by these means:

• Conduct a survey
• Interview your customers
• Read their feedback
• Launch a usability test
• Whatever means to obtain feedback

An outstanding product (i.e. a product reaches product/market fit) addresses and solves a problem or need that exists in the customers’ world. The most meaningful insight might not always be attained by A/B testings, but by learning through your customers.

Continue reading here