Photo credits Carlos Muza

A primer for getting started with product analytics

Kipkorir Kirui
Oct 30, 2017 · 8 min read

According to The Economist, oil is no longer the most valuable resource in the world; data is. A lot of companies are collecting and sitting on user data. According to my observation, only a small portion is using the data to improve their product. Why is this the case?

From the conversations I have had so far three common challenges came up:

  • There is the perception that you need a specialized resource like a data analyst or data scientist to get started.
  • Data can be overwhelming. We collect a lot and so the questions is what do should we track and what should we leave out as we can’t track everything?
  • Tools that can easily help you understand your data are said to be expensive. Building an internal tool is not a good option either.

In this article, I will share a couple of tactics you can use to get started on the product analytics journey or how you can improve how you currently use your data. The tactics apply both to startups and traditional companies irrespective of the amount of data they collect. Please keep in mind that this is not a comprehensive guide — it is a guide to help you get started.

PS: Tips and tricks imply that you can use it as a short-cut and as a result, I have intentionally opted not to use those two terms.

Step 1: Choose the right data points and metrics

It is tempting to track each and every data point in your product but this quickly gets complicated and overwhelming as your data points increase.

In my previous post, I highlighted why when I was at Eneza Education we opted for the Startup Pirate Metrics. I will expound further on this decision in this article.

Vanity metrics v/s Actionable metrics

When choosing metrics you want to know the difference between vanity metrics and actionable metrics. Vanity metrics tend to be impressive but not actionable. The guys at iacquire have broken this well in the post below.

A good way to derive your metrics and data points to track is by looking at your customer journey.

Your first task, therefore, is to document your customer journey using a customer journey map. UX Mastery did a comprehensive article on how to create your customer journey map.

Once you have your customer journey map you can then use the approach advocated by The Startup Pirate Metrics. Unlike what the name suggests, it is an approach that can be used by anyone

The Pirate Metrics structures your metrics based on five key customer activities (AARRR):

  • Acquisition — how do users find you and sign up for your service?
  • Activation — getting your users to do their first meaningful action and consequently getting their first “aha” moment. If for example, you have an app allowing users to purchase electricity tokens you would consider a user activated only after they purchase power for the first time.
  • Retention — this is pretty self-explanatory. Once a user is activated you want them to keep coming back to your product.
  • Referral — getting your existing users to invite other users to try your product.
  • Revenue — how do you make money?

This is the typical journey a user will go through for any product.

What are some of the metrics you can track in each step?

  • Acquisition — Customer Acquisition Cost (CAC), # of new users, Conversion rate
  • Activation — average time to activate, activation rate
  • Retention — active users (Daily, Weekly, Monthly), D1, D7, D30 retention, churn, stickiness (DAU/MAU)
  • Referral — referral rate, # of referrals, referral conversion rate
  • Revenue — average order size, repeat orders, revenue earned/day/month/year

The last thing you need to do before you decide on your metrics is to understand where your product struggles the most. The thinking behind this is that you want to pay more attention to where the product is struggling. As an example, there are products that struggle with acquisition but retain well while others face the opposite problem — they acquire easily but struggle to retain.

Important note: For revenue always split first-time revenue from recurring revenue and track them separately. This is important so that you can understand what is driving your growth.

Step 2 — Select the right product analytics tool

There is a good number of mature product analytics tools out in the market. The common ones are Google Analytics, Heap Analytics, Mixpanel, Kissmetrics, and Amplitude.

So how do you go about selecting a tool? A few pointers:

  • Most product analytics tools have a free tier. Before deciding on the one to implement fully it is important that you play around with a couple. It should take about two weeks for you to understand the capability of each. I would suggest you try at most three and not more.
  • What stage is your product in? Are you still validating the product, are you looking to scale it, or is it an established product? Typically, when a product is new in the market, you can’t afford to spend a lot of time thinking through this. For a product with a proven product/market fit a longer-term strategy is important. Additionally, the volume of data you collect will change as your product goes through the different stages.
  • What volume of data are you looking at analyzing? The more the data the more expensive it will be. This might mean you have to build your own analytics tool or use the off-shelf for only certain sections in the AARRR funnel. You can also decide to sample (only send a portion of the data to the analytics tool) instead of collecting all the data. This will highlight the general issues your users face.
  • How long do you intend to use the tool? Is it short-term as you build internal analytics tool or is it something you will use for a longer period? You can use off-the-shelf analytics tools to understand how to design the internal tool. The length of use also has a cost factor to it.
  • Are you only using the product analytics tool or is it complementing what you already have internally? If it is complimentary it makes sense to choose a tool that takes care of what the internal tool can’t do.

Mixpanel and Amplitude offer the most comprehensive product analytics capabilities out there. However, the two are quite expensive! Google Analytics is probably the cheapest but you will have to do a lot more to get your insights compared to Mixpanel or Amplitude.

At the end of the day make sure you experiment a lot before selecting a tool. More importantly be careful when it comes to billing as almost all analytics company will have a steep price once you use more data points than the package you paid for. To make it even worse, this is billed after usage!

Step 3—Understand how to quickly generate insights from your data

Before you select the product analytics tool it is also good to understand how you will analyze, visualize, and present the data in a way that allows you to generate actionable insights.

Most off-the-shelf product analytics tool like Google Analytics, Heap Analytics, and Mixpanel will do enough analysis for you to get started. However, at some point you will have to carry out additional analysis on your data to generate more insights.I won’t touch on that in this post.

There are many ways you can manipulate your data to generate the insights that you can immediately use to improve your product. I will share three angles you should look at when starting out.

  • Trends using graphs — You want to understand overall trends when it comes to your product. For example — Is your revenue increasing over time? Are you acquiring more users over time? Are changes to your on-boarding process improving the number of users activated? Representing your data in a way that shows performance over a specific time period at a quick glance can help you quickly know how you are performing.
  • Task completion using funnels — Funnels help you track critical steps you that your users should complete. For example — what percentage of users started and finished the registration process? What percentage of successfully registered users were activated? The key thing to note here is to use percentages and not absolute numbers as that will quickly paint the picture for you.
  • Cohort analysis to understand user behavior — a cohort is a group of people who share a common characteristic over a certain period of time. Cohorts can be based on the date the users signed up, their first purchase, purchase size, referral source and so on and so forth. Cohort analysis breaks down analysis to subsets based on common behavior. Instead of treating your users as one amorphous group cohort analysis allow you to meet user needs more effectively and optimize their experience based on similar behavior.

Bonus tip — Define your North Star Metric (NSM)

The North Star Metric is the single metric that best captures the core value that your product delivers to customers. When this metric improves the company improves. To help you better understand what NSM is let me share a couple of examples (unverified):

  • AirBnB — number of nights a guest book
  • Facebook — daily active users
  • Medium — total time spent reading
  • Quora — number of questions answered

To derive your NSM you will have to spend quite a bit of time understanding the value of your product. The article below is a good guide for this exercise.

In Conclusion

A data-aided approach can make a big difference for your product. The sooner you get started the better; don’t wait for all your ducks to line up. Keep in mind that the more you do something the better you get at it. Last but not least remember that data tells you what is happening but doing user research will tell you why. Implementing product analytics is not a reason to stop talking to your users.

How have you approached this? What worked and what didn’t work? Feel free to share your approach below in the comment section or reach out to me on Twitter.

Nairobi Design Community (NDC)

Nairobi Design Community brings together product designers…

Nairobi Design Community (NDC)

Nairobi Design Community brings together product designers, UXers', startup founders, UI designers, and any other people who have an interest in using different design thinking methods to solve our day to day life challenges. #UXNairobi

Kipkorir Kirui

Written by

Ex-child, Reluctant adult, Experience Designer, UX Researcher, Design Facilitator, Senior Product Manager, Co-founder Made by People, Product at Microsoft

Nairobi Design Community (NDC)

Nairobi Design Community brings together product designers, UXers', startup founders, UI designers, and any other people who have an interest in using different design thinking methods to solve our day to day life challenges. #UXNairobi