I’m not a big fan of data analysis. But I love to know more about users- who they are and how they use products, which ultimately and ironically only data tells me. So its a love-hate relationship. I started off being a pure “intuition” driven approach to product building. But I’v gradually moved into looking at data and then understanding it’s merits… in a really slow way! (I know a few of my colleagues who crack this. Day 0 = Lets get data driven. Day 1: Analytics Masters!)
I started in Product Management in a small startup (3 people). Since we were just starting off, we had no data and most of the feature decisions were either from experience or from instinct. That’s how any business would start off I guess.
- You identify a big enough problem (or a small one for a large volume of people)
- You validate it, avoiding cognitive bias as much as possible
- Build a solution around it
- See usage data
- Make changes
- Go to step 4
As I moved into a larger size companies, the amount of data being gathered around product usage and users started growing. This means that our users were now essentially ‘talking’ to us through data.
And when a whole lot of people start talking to you, you get a whole lot of noise and maybe some important signals.
(No relation to this blog, but do check: Signal Vs Noise and say thanks to Jason Fried & DHH for writing amazing stuff.)
Typically I went through different stages of data quantity and detailing. (We had seen a similar comparison on document sharing which listed the different documentation problem you feel face as your team keeps growing.)
Stage 1 — Anyone there yet? (Acquisition) 😵
When we started off, we identified a problem from your personal experiences or by observing someone or something. In this case, you start off with an assumption (maybe with some limited amount of data) and then build a minimum viable product. MVPs can range from a landing page / a blog / a video or even just a dummy site to get an initial understanding of whether the problem you think is worthy enough to be solved, is really one that resonates with a lot of other people as well.
We were pretty much inexperienced in gathering and understanding data, but knew the basics of tracking things with Google Analytics, so that’s what we started off with.
Stage 2 — Well that’s not how I thought you will use it! 🤔 (Adoption)
I observed the next stage of data when I worked on a side project. This started off with website traffic, sessions, time per page etc. Since I wanted to really know wtf were users doing in the product, since we hadn’t set up any events tracking, we looked at some recording tools we could use and arrived at Inspectlet. That proved to be really useful ‘cos we discovered some interesting things about our product. Like…
Users don’t care about some of the features you thought were really important. Meanwhile, they just master other features that you thought would be difficult for them to understand.
Stage 3 — Hordes 👯
At this stage ,you have a good amount of traffic coming in to your product. But it’s really too early to look specifically at “who” these users are since that number isn’t ver high. But what you can look at now is cohorts of users. How many users come to a certain pivot point/feature in your product, how many drop off, where do they drop off, why do they drop off, where are they coming from and where are they going to, are they new or repeat users?
Stage 4 — Who ARE you ? 👻
If you are step 3, you are doing pretty well for yourself! You seem to have reached a certain degree of product market fit, and you have some paid customers. But now its time to understand your users a bit more. Especially, because, well ..they are paying you! So you might as well know more about these kind of people so that you can help them better and maybe find more people like them. Getting a tool that helps you understand your users now will add more value than just looking at cohorts of invisible page views, sessions and events. The team I work with finds Mixpanel to be good in this particular area. I found the “Explore” section especially useful since it deep dives into the specifics of each user, the kind of events they triggered.
Stage 5: See you soon, ok? (Retention) 👋
Great. You have a large incoming audience. You have usage data. You have user data. Now what? Well, are people using your product once or are they coming in like the Flash and leaving like The Man of Steel? In short, what’s your retention like? Looking at retention data is priority #1 once you have a product that’s attracting a lot of users. Without any retention data, you never really know whether you have a set of users who are consistently using your product (so that eventually you can monetise them). Again, since I’v used Mixpanel — it has retention metrics that will give you a birds-eye of product as well as feature retention data.
If you noticed, I’v kind of moved through the whole AARRR metrics above:
Acquisition → Adoption → Retention are covered in this part. But I haven’t touched too much on the Referral and Revenue bit. Maybe in later editions!
The steps listed above will at least get you started on your journey to being “data driven”. There are a lot of blogs out there that will guide you into how to go about executing it, so that’s not something I’ll dig deep into. 2 reasons
- Loads of better material than I can ever create and
- I’m not the best person to be guiding you on this.
As your product grows, the amount and type of data that you need to capture will change. You will start horizontally first, capturing a high level usage data and as you grow bigger and better the level of granularity of your data should increase. Beginning at cohorts, you should move towards knowing the precise details of each and every customer of yours. That will help you understand whether the users who you are building for are really the ones using it.
