How to Get More Out of Your Observational Data

Sofia Quintero
May 23 · 7 min read

1. Back Up Quantitative Findings with Qualitative Studies

Quantitative data can be effective for figuring out the “what” of a relationship. It can tell product managers how often people are visiting a particular web page, or how many of the people who buy one product also buy another.

2. Beware of Measurement Effects

In some cases, the act of gathering the data itself will affect the results your product management team gets. People who know they are being watched might be less likely to buy a product that has a social stigma attached to it, or to express their true preferences. For example, people tend to over-report their income and how often they exercise.

3. Start With a Falsifiable Claim

It can be tempting to avoid prejudging your data by trying to begin the process without any expectations for what that data will show.

4. Make Sure Your Model is Not Too Smart For Its Own Good

In general, the simpler your data, the less complex your model needs to be. Where the sample is large and where the relationships of cause and effect are straightforward, linear models will do the trick. When this is not the case, your product management team will need to move to more complicated tools.

Data is Not Neutral

With observational data getting ever more plentiful and reliable, it is critical that product managers put systems in place to distinguish signal from noise. The bigger and more complex your data set, the more important those systems are.


Hungry for Insight

A community for product people leading change

Sofia Quintero

Written by

Founder and CEO at https://getenjoyhq.com/

Hungry for Insight

A community for product people leading change