On data-driven product design: The battles behind using data

How do you know if you are truly data-driven?

How many customers do you need to talk to? How often? How many A/B tests do you need to execute? How much do you need to invest in user testing sessions? How many design sprints do you need to run? How often do you need to check your analytics?

It’s very easy to preach to the data-driven choir: “You should be talking to customers on a daily basis”. ‘You should be going to the next sprint planning with data not opinions”. That all feels like sound advice and it is, but very few people act on it. In my experience there are many reasons why. Most designers and product managers I talk to are facing at least one if not many of the following problems:

  1. They can’t access customers for research interviews because of internal politics.
  2. When they get to do any type of research they struggle to get the product team to pay attention or incorporate the learnings.
  3. They don’t have access to systems that gather customers’ interactions with the business eg. sales systems, customer support tools, NPS surveys etc.
  4. When they have access to all the data they need, their data is scattered so they spend more time copying and pasting information from one tool to the other than analysing it.
  5. They want to do more qualitative research but management only cares about quantitative analysis.
  6. They may feel insecure about their abilities to look at data and extract meaning from it.
  7. They feel comfortable only with a couple of research techniques and as a consequence they become victims of the law of instrument: “If all you have is a hammer, everything looks like a nail”. In order words, let’s JTBD everything!
  8. They feel they are constantly rushed into producing meaningful insights from a very “lean” research process.

If you identify with any of the challenges above, I’m sure you can probably add a couple of items to this list.

The truth is that not many people went to work today knowing that they would have all the data they needed, in the right format within the right tools, just waiting for them. Being data-driven for most people feels like a series of battles that only the most resilient can win.

You have to pick your battles with internal politics, lack of resources, lack of knowledge and lack of support among other things but that’s ok — it’s part of the job and it needs to be done.

I’m always inspired by all the product people I get to meet every week. They pick their battles. They fight and they push through, but they don’t always know that their struggle is not unique. From the tiny startup to the very large successful company, they all have to find ways to make the most of the data they have. They all have to hack their tools and rally their teams around a more customer-focused, data driven path. The problems are the same but at a different scale.

“Data breadlines suffocate organizations. Teams don’t receive timely answers to their questions or any answers at all. Data analysts don’t leverage their expertise to its fullest extent. Teams make uninformed decisions. Consequently, the company never realises the true value of its investment in people or data.”
Winning with Data by Tomasz Tunguz

So, how do you know you’re data-driven?

Maybe you are because you haven’t suffocated yet. You’re still pushing to get access to the data you need, to get the best tools, to build the best processes, to learn as much as you can about researching, iterating and experimenting. That’s how you know you are data-driven.

Are you great at being data-driven?

That’s a different question.

In the same way that being money-driven doesn’t mean that you have a ton of cash in the bank. It just means that your motivations are aligned with having money. Being great at using data to make better decisions and create value has more to do with your ability to consistently learn and improve your outcomes, based on informed decisions. It has to do with your analytical skills and the willingness to execute on your findings.

I’ve met brilliant product people with the right skills struggling to get the data they need, and I have met people with loads of data at their fingertips but not sure what to do with it. Both are battles worth fighting.

Pick your battles.