Stop just looking at your Net Promoter Score! A better way to utilize NPS
NPS is tricky. In my business consulting days, we loved and highly promoted NPS. Our clients, who were usually C-suites, tended to buy in easily as well because it’s simple, straightforward, and proven to be correlated with revenues.
As more and more companies started to adopt NPS and use it as their North Star Metric, I also see more criticisms. Interestingly enough, I have found that UX researchers are especially no fans of NPS. Some have critiqued it in a very constructive and systematic way. Some had a good time making fun of it.
Is NPS really that useless?
A few months ago, my product team decided to abandon NPS because they could not see a clear relationship between NPS and MAU (Monthly Active Users), which is our primary KPI.
I found that many expect NPS to work like a magic number. People watch it closely on a daily basis and get emotional when it changes, hoping that as it rises, more engaged users and profit would follow.
However, tracking the number itself does not give you too much meaningful information. The key is to find the “driving factors” behind NPS.
In short, a better way to utilize NPS data is to find out what made someone an extreme detractor (those who gave you 0) or promoter (those who gave you 10), and test those factors against the passives to see if the improving the factors can move the needle (change the NPS they give). To do so, you need a combination of qualitative and quantitative research.
A step-by-step guide
1. Map out the entire user journey using 1:1 interviews
First, recruit both promoters and detractors and set up 1:1 interviews with them. In the interview, draw a timeline and have the participant walk you through his or her entire journey with the product/service. The goal is to map out everything that happened from the very beginning — before they even got to know the product — all the way to now.
Ask participants to recall major events and triggers along the way and what NPS they would give in those particular moments. Note that one’s interaction with a product/service is usually across multiple channels, and sometimes might not be directly related to the product/service itself. A trigger could be word-of-mouths from a trusted friend, or an intriguing advertisement in the subway.
Tips:
● Before the interview, form a hypothesis of what the journey/timeline might look like. During the interview, pay attention to the gaps between your hypothesis and the information gathered from participants. Dig deeper into those gaps and find out the reasons behind them.
● Jog the participants’ memories with the aid of such items as brochures, marketing materials, registration pages, etc.
● If the participants have a hard time recalling events in a chronological order, start with the ones that are the most salient and memorable to them.
● During the interview, focus on facts and the users’ actual behaviors, rather than opinions.
● This timeline could end up looking like the timeline in the jobs-to-be-done framework. Feel free to incorporate its diagram.
2. Synthesize triggers and events into driving factors
By the end of the interviews, you will get a lot of different triggers and events. The next step is to organize and group them into broader categories. For example, an event where one “had a car accident, called the insurance company and got transferred three times until the right person picked up the phone” may be broadly defined as “accident assistance”. Note that you might still want to have some granularity here. It could be too broad if you define it as “customer services” since customer service includes general inquiries, roadside assistance, opening new accounts, etc., and customers might have different experiences with each.
3. Run a survey. Gather overall NPS and NPS for each factor
After having all factors concluded, we need to have a holistic view of how the company is doing in each factor. Run a survey with your existing customers. In the survey, ask for the overall NPS first, and then for each factor, as shown below.
Essentially, you can see the first question as relationship NPS and the rest as transactional NPS.
*For the difference between relationship and transactional NPS, check out this article.
4. Calculate the correlation between the overall NPS and the NPS of each factor
The final step is to analyze and utilize the data collected from the survey. Calculate the correlation between the overall NPS and the NPS of each factor. The goal here is to understand what factors might drive the overall NPS.
Once you are done with the correlation calculation, create a 2x2 matrix, and plug the factors into the matrix based on their current NPSs and the relationships with the overall NPS. This helps you see where the issues are and prioritize what to tackle first. Ideally, you want to start by working on the factors that are highly correlated to the overall NPS, but currently have lower NPS.
No qualitative researcher?
It is also possible to identify NPS driving factors with data. You can look into the variables below (depending on what user data you have), and run a linear regression model to determine which variables are correlated with NPS, and to what extent.
Who — who is the user who gave this NPS
- Country, industry, product licenses, tenant information, etc.
What — What does the user do with the product?
- Retention, frequency, features used, amount paid, etc.
Context — Under what context did the user rate this product?
- Mobile vs. web responses, activities within the last 24 hours, etc.
Do you have other ways to better utilize NPS ? Would be great to learn more about yours too!
