May the Force (of Your Data) Be With You

Product organizations talk about how data is the heart of their decision-making process. To learn about it, last week I attended a meetup on ‘Product Analytics’ hosted by HelloMeets. Sahil Pruthi, Product Manager at TravelTriangle, shared his experiences on how exactly Product Managers strategize the roadmap. He discussed the methods/metrics used to analyse the data to make the decisions. We discussed what is a good framework to help one think through the entire process.

Being in Support (working as Technical Consultant at Kayako), this session helped me to understand how Product Team works on designing and implementing new features roadmap.

What we discussed during the session?

Everyone knows analytics are important for product managers. But not everyone knows why they are important and even necessarily what they are. Hence, the session was divided into categories to understand the details.

· Know your funnel

· Bucket your features

· Product Metrics

· Solve one block at a time

· Tools for experiments

Know your funnel

Funnel refer to the moments in the customer journey. Hence, it starts with when your user starts with your product as a visitor and ends up being loyal to you. The first step is to synergise customer journey with the stages and their priorities.

The Funnel looks like:

Try to define all the stages and the necessities for making your customer journey smooth:

· V-L stage (a new visitor visits your site)

· L-A stage (visitor has shown interest in your product, trial customer)

· A-C stage (trial customer gets converted to a paid customer)

· C-R stage (loyalty pays here, your users refer your product to others)

Bucket your features

There are three categories used to define the features to know in which bucket they follow and which stage they belong to. However, the main thing is, you need to decide that X feature will impact Y metrics by z% in one-quarter (or months).

Necessary — the mandatory features for increasing product enhancements and the most voted functionalities. Necessary features are the ones which bring money.

Hygiene — the features for closing a trust of your customers. Like FAQs, smooth onboarding, cancellation process, etc.

Experimental — where you’re not sure how well these features will perform. A/B testing needs to be done for experimental features.

When you know the categories of the features, it is very important to know how to set the Priority of the feature. The formula for feature’s priority is:

Product Metrics

After knowing the funnel and features bucket, it is important to work on metrics before you start the process. There are three types of product metrics:

Project Metrics — immediate product metric

Like the metric which increases due to the change you implemented (more clicks on a product link by a change in UX)

Lead Metrics — subsequent metric increase

Like increase in numbers of visitors or lead funnel after changing the UX

Lag Metrics — which is involved in A-C funnel i.e. business money

increase in “GMV” gross merchandise value (more paid customers)

Benchmarking — techniques to know how to start

Before bringing any change, Think of a project metrics — quantify the impact of a change — decide a metric to be achieved. Benchmarking is done on the basis of:

- Previously launched product/feature

- Existing product data

- A/B testing — experiments

- Plain Guesstimate

- Competitor analysis

Solve one block at a time

Always focus on one block/metric at a time. Don’t jump into too many things. The key point is, whether you over-achieve or under-achieve, you need to emphasize on the decided metric rather than other metrics.

Tools for experiments

Launching 100% product feature to all the customers won’t give you a clear picture. You will not be able to get the correct –

• Data
• Learnings

The best way to launch a new experimental feature is, divide your audience 50–50%. Route the old feature to X% of the users and the new feature to 100-X% of the users. Collect the data and then analyse it to gauge whether experimental feature shall be considered as a necessary feature or it shall be dropped (per the data).

Experimental features are the ones which are converted as Necessary features if they help and increase the conversation rate. However, there’s one more interesting thing involved i.e. RCA — root cause analysis. RCA says to re-read your funnel and analyse it, going reverse. You need to know the variable factors and quantify them.

Over-achieving and under-achieving — both are important for Learnings. You need to take the learnings into account while factorising any functionality to be introduced.

There are a few tools which can be used to collect and analyse the data:

· Google Analytics

· Firebase

· Hotjar

· Mixpanel

· Clevertap

At last, when you know which features are getting more %age among all and your customers are also involved in it, ta da — that’s it! Time to take the decision and make this experimental feature as a necessary feature.