In defense of NPS (Net Promotor Score)

Mike Donahue
Jun 5, 2018 · 11 min read

In the article Net Promoter Score Considered Harmful, UX expert Jared Spool refers to Net Promotor Score (NPS) as harmful, vacuous, and built on wacky science. It’s none of those things. NPS helps a company sense and respond to users, it’s rich in insightful data, and it’s built on brain science.

Jared goes to great length to prove the inherent flaws in using NPS to predict customer loyalty, future behavior, or business growth. On this point I agree. However, the supporting evidence he gives about why and how NPS is flawed is itself flawed. Jared’s article frames the NPS score as static, absolute, and the sole indicator to look at to determine success. It omits crucial details about, and behind, the NPS method. In this article I’ll fill in the information gaps and explain why they matter. This is to help others make an informed decision about whether NPS is a useful technique for their business.

First let me recap what Jared perceives as the flaws with NPS:

  • NPS is unreliable at predicting the long-term future loyalty, future behavior, or business growth
  • NPS uses “wacky science” as a formula
  • NPS hides success
  • NPS question is “garbage”
  • NPS can be gamed (I agree with this, but anything can be gamed)

A note: Jared only referenced Fred Reichheld’s Harvard Business Review (HBR) article (The One Number You Need to Grow, 2003). There are lot of details you won’t find in that HBR article. Details you will find if you read The Ultimate Question 2.0, also by Fred Reichheld. Details that I’ll share here because the details matters.

The psychology behind NPS, and why it matters

As I said, I agree that NPS is unreliable for predicting long-term loyalty, future behavior, or business growth. It is however reliable at revealing how your users perceive your brand, company, products, or services today. In other words, you can predict the likelihood of near-term loyalty, behavior, and growth. If you understand what happens under the hood of NPS.

Let’s look at this through a different lens for a moment. Imagine someone you know asks you to recommend them for a job where you work. What do you do? What’s at stake when you give a recommendation for this person?

The short answer is, your reputation is at stake. This matters because, if you give a recommendation for someone and they ends up a bad fit, it’s your reputation that takes a hit. Your peers will question the quality of your recommendations in the future. The premise of NPS follows similar thinking.

Let’s begin by examining a typical NPS question. “Based on your recent experience, how likely are you to recommend {insert product, service, or business} to a friend or colleague?” Breaking down the psychology of the question is critical to understanding the effect it has on how we process our response.

Notice that the question doesn’t ask if you’re likely to recommend to complete strangers, or give an anonymous review on an e-comm site. It matters because it asks you to recommend to those within your circle of trust, friends and colleagues. This means putting our reputation on the line with people you’re close to. Reichheld cites our willingness, or lack of, to put our reputation on the line as a signifier of potential loyalty, and by extension, a predictor of future behavior and business growth. He’s kind of right.

In many situations this logic is sound, but there are a few reasons this logic fails to hold up in business:

  • Unlike friends, new options for better/cheaper products and services pop up everyday.
  • We trust that friends will show loyalty in return, it’s rare we trust business, products, or services will do the same.
  • Similar to friends, over time your relationship status will change based on how consistently the other side prove their loyalty to us.

It matters because we’re cautious when our reputation is at stake. We have a biological need to be perceived well by others. We’re unlikely to risk that when we’re uncertain if the other side will continue show their loyalty to us. Or uncertain they will continue to deliver the quality of experience that we’ve come to expect.

The recommendation is great, but not the real point of NPS

Jared points out that it’s hard to know if someone ever gives a recommendation. The reality is a person may never have the chance or need to make the recommendation. That doesn’t make the gesture useless or meaningless. While it’s good to get the recommendations it’s not the only indication of success.

NPS, at least in theory, attempts to find out if people believe they are willing to put their reputation on the line. It’s well known that people are unreliable at self-reporting. But people do know how they feel in a given moment. And they will go to great length to protect their reputation.

An individual can tell you right now, with certainty, how willing they are to put their reputation on the line based on recent experiences. Whether they have the chance to make the recommendation is uncertain. This matters because the individual has still indicated how likely they are to put their reputation on the line. Getting the actual recommendation is icing on the cake.

Jared makes a case that NPS is just exposing how satisfied the user is. That’s kind of true but NPS goes deeper than just satisfaction and I’ll explain why and how in the next section. He points out that if you’re measuring just satisfaction, use a C-SAT survey.

I’m not going to debate NPS vs C-SAT because each tool is intended for similar, but different, purposes. Much the way a carpenter has different types of hammers, saws, and screw drivers, we need a diverse set of tools that are tailored to specific tasks. Think of NPS and C-SAT for instance as two types of hammers. Each one gets at how users feel, but each do it in a different way.

The psychology behind the formula, and why it matters

Before I get into the psychology of the formula, or the ”wacky science” as Jared puts it, let me recap how it works.

NPS uses an 11 point rating scale, from 0 to 10.

  • Ratings from 0–6 = Detractors (won’t put their reputation on the line, even if given the chance to)
  • 7–8 = Passives (may or may not put their reputation on the line if given the chance)
  • 9–10 = Promotors (believe they will put their reputation on the line if given the chance)
11-point NPS Scale

The score is calculated by subtracting the % of Detractors from the % of Promotors. A score of 100 is the best you can do and –100 is the worst. The numbers however are only there for the purpose of calculation. It’s the labels at each end of the scale that tells the story of how people feel.

At the negative end of the scale we have the label ”Extremely unlikely and on the positive end ”Extremely likely. The simple addition of the word “extremely” impacts how our brain processes our response. The word extremely moves process from binary and logical to variable and emotional. It matters because it forces us to consider the intensity of our feelings about the experiences. This goes well beyond just satisfaction. Beyond a simple “will I or won’t I.” It’s a far cry from asking “Would you recommend us? Yes or No.”

NPS is a scale of emotional intensity, not numbers

NPS scale with emojis that depict the emotional range from extreme frustration to extreme delight.

I’ll take the scale one step further. If we add emojis to the scale it become more obvious this is scale of emotion, not numbers. On the positive end is an emoji with a great big smile and two thumbs up. If you asked a real person with that expression on their face how likely they are to recommend, chances are you get an enthusiastic yes.

On negative side is an emoji with a scowl and furled brow. If you asked a real person with an expression like the one above 0, the same question, chance are you get an equally enthusiastic no. Maybe even, “No f’n way in hell mister!!!”

What about the middle of the scale, or even half way between the middle and the top? How would a real person with an expression like the one above 7 or 8 answer? They may or may not give an unenthusiastic response either way. It matters because it’s not about the number, it’s about the intensity of how they feel. This is brain science, not wacky science.

When we consider the scale in terms of emotional intensity rather than just a number we see that the distinction between ratings is not arbitrary. Acknowledging the correlation between the ratings and emotional intensity is important. It matters because loyalty is based on emotion, not logic. More over, we’re only loyal to people and things that we have a strong positive feeling about. Looking at NPS in this way reminds us to focus on the perceived value from the users’ perspective and not a score from our perspective.

NPS is a reliable predictor of near-term success

While NPS isn’t great at predicting long-term success or behavior, there’s a strong correlation to near-term success. We need only look beyond the score as static, absolute, or the sole indicator. This is a critical flaw in Jared’s article for three reasons.

The score is variable, not static

While a score represents a moment in time, we need to look at it over time. A fact the Jared points out in one place but ignores in others. We need to compare present to past to see how the score is trending. Is it going up, down, or is it unchanged? The point is NPS is not a one and done metric. Comparison gives us a view of the score relative to other factors.

It matters because looking at it over time we can look for causality. Something that we can’t find if we look at the score as static. To be clear, it can’t guarantee we’ll find the cause, but it will show us if we’re heading the right direction or not from the customers perspective.

The score is relative, not absolute

We also need to compare our NPS to our competition within our industry. Jared gave a scenario were we received an NPS of –40. Any which way you slice it, –40 is bad. That said, if your competition has an NPS of –60, you’re going to do better than they do in your market. In this way NPS can predict near-term loyalty, behavior, and growth. At the very least, it will show a strong correlation.

Keep in mind that NPS is relative across industries. Let’s say you have an NPS of 40, not –40. Is 40 good or bad? Well, it depends. If you’re in hospitality (like a hotel or restaurant), it’s abysmal. On the other hand, if you’re a cell phone carrier with a 40, break out the champagne. It matters because we grade on a curve. That’s why a cell company with a 40 is a rock star and hotel with a 40 goes out of business.

We use all of our experiences, not just our experience with your product or service, when we give our rating. The key is to always improve your overall NPS, and make sure it’s always better than your competition. NPS will have a strong correlation with success today, but tomorrow is a different story.

The score is one indicator, not the sole indicator

Jared depicts a scenario where overtime the ratings increase, they go from all 0s (zeros) to all 6s, but the score remains the same, –100. He asserts this hides success. That’s true if you only look at the score. That’s just lazy and ignorant.

Let’s look at this through another lens. In football, the only number that matter’s to the casual fan is the final score. It tells us if our team won or lost. Coaches and analysts on the other hand look at the numbers behind the score to understand why they won or lost. They know that in order keeping winning, or prevent losing, they need know what contributed to the outcome. For instance, they look yardage gained by the offense and give-up by they defense. They track this data over time to look for trends. The final score tells us the outcome but not what likely caused it. Looking at the data behind the score is critical step to uncover possible causality.

It’s like Bruce Lee tells his student in the movie Enter the Dragon. “It is like a finger pointing the way to the moon. Don’t concentrate on the finger or you will miss all the heavenly glory.” If you focus too much on just one detail you miss the bigger picture.

Like a score in football, if your score is better than your competitor, you win. However, if you only look at the score you may miss crucial insights. It matters because morale is important to your team. Going from all 0s to all 6s shows a significant improvement. They need to know that their efforts are making a difference. Not every insight is visible on the surface, dig into the details behind the score.

Anything can be gamed

Finally, one of Jared’s lesser points is that you can game NPS. That holds true for most anything. What’s more, if you’re not careful, your own biases will cause you to game yourself. By that I mean, you could frame your question in a way that guarantees better scores.

I’ve heard Jared say, “What get’s measured gets done. What gets rewarded gets done well.” So true. When people’s jobs hang in the balance based on a single score, they will find ways to raise the score. In the book The Ultimate Question 2.0 the author recounts how a branch of rental car company tossed out surveys with poor scores in order to report a better overall NPS. It happens when NPS is misunderstood and only the score is the only focus.

It matters because it’s up to you and your organization to implement NPS in a healthy way. Done wrong, you will not gain realistic insight into how your customers perceive you. This will in fact do you and your business more harm than good. Done well, NPS is a reliable way gain useful and meaningful insights. To hear how users perceive your brand, products, and services today and over time. This is how you succeed in the near- and long-term.

UPDATE: I cleaned up some typos and grammatical errors. I also want to add one note that I ommitted when I orginally posted this article. That is that when conducting an NPS survey if you just ask for the score you miss the chance to gather qualitative insight. Follow up your recommendation question with an open text field and ask the respondent reason for the score they gave you. Expect that the detractor will be much more forthcoming with “feedback.”

Bottom line

  • NPS is neither harmful nor vacuous nor wacky science. While it may not predict loyalty, behavior, or growth, it can show you how you compare to your competition and your progress over time.
  • NPS measures emotional intensity over time. Whether there’s a direct correlation between NPS and loyalty is up for debate. What isn’t in debate is that loyalty is based on an intense positive connection with someone or something you trust. No other method I’m aware of attempts to measure this.
  • NPS meets Jared’s own criteria for a “useful” business metric: it is easy to measure; it produces a number we can track; and it feels legitimate — NPS doesn’t just feel legitimate, it is legitimate.

Talking UX

Stories, thoughts, and opinions about any and all things UX.

Mike Donahue

Written by

UX Ideologist, Radical Inclusionist, Designer, and Photographer

Talking UX

Stories, thoughts, and opinions about any and all things UX.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight.
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox.
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month.