Beyond CSAT and NPS

Why your customer survey is holding back your business

Consider the following experiences:

As your waiter picks up the check at a local restaurant, he gives you a card and asks you to please let them know how your meal went.

While browsing a retail website, a popup interrupts you to ask for feedback on your online shopping experience.

You call your insurance company and before you’re helped, an automated phone system asks you to stay on the line after the call to answer a quick survey about their service. The next day you get an email nagging you for feedback.

The cashier at the grocery store hands you the receipt and points to a jumble of letters at the bottom, explaining that if you use the code to go online and fill out a survey, you’ll be entered for a chance to win something that you won’t remember because you’ve already stopped listening.

In today’s business world, no one escapes post-service-surveys (PSS). It’s not for lack of trying — marketers know all too well that most people immediately click the X, hang up, delete, ignore, and throw it in the trash. Many companies spend serious money (and seriously irritate their customers) trying to get enough responses for a statistically significant customer satisfaction figure (CSAT) or net promoter score (NPS).

Businesses have come to rely on these tools to manage their customer service operations, and the reasons for it seem obvious. The metrics correlate with the outcomes they care about most, like customer retention and revenue. They provide an accurate and objective view into operations on an ongoing basis, so managers can track progress constantly and solve problems as they arise. Even better, the surveys help build strong relationships with customers by collecting first-hand feedback and showing that the company cares what they think. Right?

This logic is dangerously flawed. A closer look at each of these ideas will show how this path, despite its popularity, leads away from the ideal customer experience instead of towards it.


The Ugly Truth of Post Satisfaction Surveys

Correlation is an irresistible attraction, but it can easily be a trap. Statistics 101 teaches us that correlation does not imply causation — it can only suggest where to look (Pearl, 2009). So, when managers analyze the relationship between marketing numbers, like CSAT, and objective business outcomes, like revenue, they should take the results with a sizable grain of salt. There will almost certainly be some correlation, but that does not mean that maximizing CSAT will cause the highest possible revenue growth. Of course, common sense tells us that the quality of the customer experience is an important driver of business success. However, statistical correlations reveal nothing about how or why this relationship works for a particular company. Relying on them to drive strategy can start a wild goose chase after meaningless targets, like 100% CSAT, in which everyone forgets that it’s not the metric but the customer that matters.

“By the time managers notice something is wrong, there is already a major problem that has gone undetected for days, weeks, or even months.”

This would be true even if PSS provided an accurate, objective, and timely picture of the customer experience, but they do not. Like any voluntary-response survey, they are vulnerable to many statistical problems, including low data quality, poor instrumentation, sampling bias, and acquiescence bias. As the lens through which business leaders view the customer experience, surveys are at best foggy and at worst terribly warped. In addition, they are lagging indicators of operations quality in both production and delivery. Customers don’t provide feedback until after their interaction with the company is over, perhaps quite a while after, and more time passes before the data are processed and viewed. By the time managers notice something is wrong, there is already a major problem that has gone undetected for days, weeks, or even months. Worse, the numbers alone say nothing about the source of the problem or how to solve it.

How to abuse your customer, courtesy of mcdvoice.com

On top of all that, PSS create a dysfunctional relationship with the customer. In a transaction, the customer has one job: to pay the bill. It’s not their responsibility to set the bar for excellence and evaluate whether their experience reaches it — that’s on the company — but it’s exactly what surveys ask them to do. Customers balk at this. Nearly half would prefer not to provide feedback at all (Figure 1), which leads to the use of tacky and cumbersome incentive programs to get a response. The idea that surveys make customers feel heard and appreciated is an illusion as well: 47% say that surveys reduce their inclination to buy from the company again, and only 16% say the opposite (Figure 2). Perhaps the greatest danger of PSS is the passive mindset they create around customer service. The customer’s expectations set the standard for service quality, and the company becomes reactive instead of proactive, waiting for external feedback to tell them when and how to improve.

Figure 1 (Source)
Figure 2 (Source)

A Different Perspective on Quality

AMichelin-starred restaurant, like Quince in San Francisco, does not rely on its clientele to define excellence. A visit to the back of the house will show a kitchen in impeccable order, full of chefs that coordinate seamlessly and taste the food constantly. Before the restaurnt opens for the day, the waitstaff diligently study and sample the menu. During the meal, they serve attentively but unobtrusively — watching for the right moment to refill the water or clear the plates, checking faces for signs of delight or displeasure, and surreptitiously refolding the napkin before the client returns from the restroom. In such a place, where customers experience quality of the highest order, you will never find a survey. They know long before you’ve signed the check that you had a spectacular evening.

Chef Tusk, Quince Restaurant

This is an entirely different mindset, built on the idea that to understand the quality you’re delivering, you must observe it first-hand. The idea is not new, and it does not only apply to small businesses based on physical interaction, like restaurants. It’s the foundation of Toyota’s production philosophy, Genchi Genbutsu, meaning to “go and see.” Toyota managers and engineers learn that to design products and solve problems, you must go to the place where they happen — the factory floor — and observe as closely as possible. What’s more, everyone is responsible for quality, not just a select few. Even a janitor has the right to shut down the production line if he spots trouble, because even though this can cost millions of dollars, the loss is inevitably less than the expense and brand damage of a recall.

“…to understand the quality you’re delivering, you must observe it first-hand.”

A service quality philosophy based on direct observation and shared responsibility can work for any company, large or small, digital or physical, startup or mature. For those dependent on PSS, making the transition takes dedication, but the results are worth the effort.

Making the Change

The first step is to establish the mindset by teaching employees to proactively observe and take ownership of the quality of their work. Most companies already have some kind of training in place for quality assurance and customer service. The key to making this transformation is to overhaul this training and expand it to everyone who touches the product or the customer. They should learn what excellence in their job looks like and why achieving it is good for everyone — the customer, the company, and the employees. Most importantly, they should gain the knowledge and power to judge their own work, ask for help, and make suggestions. When employees have responsibility for their work, the tools to do their jobs well, and a sense of mutual commitment to the team, the company can take its first steps down the path to excellence.

“…companies should be spearfishing: looking for complaints and early warning signs of problems to come.”

For most businesses, this means that the nature of employee training must fundamentally change. Rote learning of rules and instructions is useless. Even the “learn-do-teach” model, often cited as the best way to master a skill, is not enough. The problem with this process is that it’s finite — once you have learned the basics, practiced in the real world, and taught it to someone else, there is nowhere else to go. True excellence means continuous improvement, not stagnation. To achieve that, the “learn-do-teach” model needs a feedback loop based on observing the work and re-imagining how it could be done better (Figure 3). Notice that observation is part of every step, providing a constant reality-check on what works well and what doesn’t. The power of this model is double: not only is it an effective method for training employees, but it also teaches them to continue thinking this way even when formal training is over.

Figure 3: PartnerHero’s improved “Learn-Do-Teach” model

Once the new quality mindset has taken hold, the management’s job is to monitor what truly matters and keep raising the bar. Instead of casting a broad net for feedback, as surveys do, companies should be spearfishing: looking for complaints and early warning signs of problems to come. This way they can ignore the noise and focus their resources on eliminating real liabilities. They should also pay attention to metrics that are meaningful, like defect or error rates, customer lifetime value, and sales. These say more about whether you’re satisfying the customer than any survey can. They also point the way to incremental improvements that make a real difference — what can your employees do to create a product so perfect, delivered with such panache, that customers keep coming back for more and bringing their friends too? Executives should focus their energies on answering this question and leading their employees to make that vision a reality.

Overcoming Common Mental Roadblocks

In the end, the biggest obstacle to achieving excellence in customer service is entrenchment in a survey-dependent management style. Moving away from the familiar can feel scary, but there is truthfully little to fear. The following are some of the most common reactions from those who doubt that this method can work for their business.

“I’ll be blind if I just throw away my metrics!”

The reality is that even with the metrics, you are already blind. They don’t show you the true customer experience as it’s happening — they only give the illusion of doing so. In many ways, that’s more dangerous than having nothing because it creates a false sense of comfort and can lead you to dedicate time, effort, and money to a customer service strategy that won’t work. Better to destroy the illusion and start searching for the truth.

“I can’t trust my employees with that much responsibility!”

Especially for companies that are growing or already large, the thought of spreading control over the customer experience among so many people, so far from the leadership, is daunting. However, the fact is that it’s already done. For your customers, the employees they interact with are the face of the company. Wouldn’t it be better if they had the knowledge, tools, and motivation to do the best job possible? Granted, some may not be ready to take ownership of their work, but they will become apparent quickly, making it easier to build an outstanding workforce. You may be surprised to find that most will step up to meet higher expectations when they are empowered to do so.

“It’s much more work to observe directly — the surveys are more efficient.”

Yes, there is some work required to transform the quality mindset of your company, but is it more work than what your PSS program requires? Many companies put significant human and financial resources into creating surveys, collecting responses, and processing the data, only to get low-quality and potentially misleading information. Re-allocating that effort to building a new philosophy of quality will pay far greater dividends.

“Without a survey, how will I get feedback from my customers?”

Most customers don’t want to respond to surveys, but many are willing to provide feedback of their own volition. Make it easy for them to contact you with complaints, suggestions, and compliments, not just through public channels like social media and product review pages, but also through a private message system. Most importantly, listen closely to those who want to communicate with you. And don’t forget that your front-line employees are getting continuous, real-time feedback from the words, tone, and body language of their customers.

Garbage In, Garbage Out

This is not a tirade against using data to inform management strategy. It’s a reminder that the old computer programming adage “garbage in, garbage out” applies to human decision-making as well: bad information leads to bad decisions. That’s why it’s so important to recognize that PSS does not provide the accuracy or richness managers need to craft a superior customer experience. It’s time to replace this paradigm with one that recognizes the distributed nature of quality control and harnesses the power of true teamwork.

“…any kind of company can use the principles of observation and shared responsibility to create the kind of customer experience that other businesses envy…”

Again, this is not novel — it’s the same philosophy you’ll find at all quality-obsessed companies. Toyota and Quince are just two examples, but any business that consistently dazzles you probably belongs in the same category. Amazon not only tops online retailers in selection, speed, and ease but also actively anticipates customer complaints before they happen. The Ritz-Carlton and The Four Seasons both laser in on the details of the customer experience to make every client feel relaxed and pampered throughout their stay. Trader Joe’s trades the typical grocery store staff structure for an interdependent team, enabling them to provide the highest-rated grocery shopping experience in the country. The family-owned tailor shop where I bought a custom suit insisted on three separate fittings to ensure that I would have a garment that fit perfectly and lasted a lifetime, just as he learned from his father before him.

The point is that any kind of company can use the principles of observation and shared responsibility to create the kind of customer experience that other businesses envy and that will keep your clients coming back for more. The first step is to throw the surveys in the trash and look around with your own eyes.


About the Author

Shervin Talieh is the founder and CEO of PartnerHero, a global operations outsourcing company focused on serving startups. He is also the product manager for Aprikot, a peer feedback solution designed to improve quality within customer support teams. He first started thinking about the need to change quality and engagement metrics in 2014, when he offered up his thoughts at UserConf, Chicago.

Shervin has three kids, two dogs and a partner who remind him how his own QA scores still need work. He recently relocated to Boise, Idaho, home of PartnerHero’s North American Operations Center.

About PartnerHero

PartnerHero provides remote teams for growing startups. We find, hire and develop our employees from around the world, and match them with the startups that could most benefit from their skills. This new form of outsourcing allows startups to have a much more direct relationship with the teams working for them, while still maintaining operational flexibility, lower OpEx, and improved functional metrics.

About Aprikot

Aprikot, formerly known as Sidekick, is a software-as-a-service (SaaS) solution for customer support teams. It operates as a brand, product, and team within PartnerHero.

Today, Aprikot connects with Help Scout and Zendesk and allows customer support agents to provide feedback and quality scores to their peers. Traditionally, customer support teams have built complex and archaic spreadsheets to track agent QA scores, using internally developed rubrics for setting scoring guidelines. Aprikot’s mission is to change QA from a punitive, scary, top-down process to an easy-to-use, peer driven experience that helps up-level the entire team.