Key Experience Indicators: How to decide what to measure?

Tomer Sharon
Jun 30, 2018 · 4 min read
Image for post
Image for post
When you measure smaller parts of the experience (only important ones, not everything), you get an indication of what specifically works well and what doesn’t. You can then dive deeper into the problem areas to understand why.

Chef Antonella opened her new restaurant six months ago. Everything has been going really well, it seemed. Revenue is up and to the right, tables are reserved in advance, and the restaurant is full and lively every single night. Business metrics show a very positive picture.

Chef Antonella also measures two key experience indicators she cares about. These paint a different picture. Close to 100% of customers are new. She has almost no returning customers. Second, on average, 70% of food is left on the plates. Something is wrong although business is great.

🔑 Why measure experience?

  1. KEIs provide information to decision makers.
  2. KEIs precede or predict business outcomes.
  3. KEIs give insights into qualitative findings and customer anecdotes.

🔬Considerations of KEI measurements

Image for post
Image for post
Results of experience-related measurements for three alternative designs. Design a is better than b and c, yet you don’t know why until you make an effort to learn.

Good vs. bad: It is not always clear whether KEIs are showing a positive or negative phenomenon.

Can you tell if these are good or bad?

  • Average time to first click is 49 seconds.
  • Average time on site is 27:38 minutes.
  • Homepage bounce rate is 99%.

Attitude vs. behavior: what people say does not necessarily match how they behave. People’s attitude is interesting yet behavior is much more telling. Give more attention and weight to behavioral rather than attitudinal metrics.

Chef Antonella’s restaurant is providing great coffee. Yet if you ask one of Antonella’s customers to rate her satisfaction with coffee immediately after she found a mistake on the check, coffee might be getting low scores. Coffee consumption, however, measured in the same exact time, might tell a more realistic story.

Image for post
Image for post
Beware not to measure behavior through attitude. Don’t ask about behavior in a survey. Instead, measure it directly.

Overall vs. specific: Measuring specific phenomena gives you the ability to focus efforts and save resources. When you measure NPS or overall satisfaction, you get scores that do not help you understand what goes well and what needs fixing. If you get a score of 66 for your product’s NPS, do you know why you lost 34 points?

Alternatively, when you measure a lot of smaller experiences, for example, at a hotel, you may find that while your overall UX score is 93%, you lost 7 points because coffee, room cleanliness, and the online checkout process got low scores. You wouldn’t know why and yet you can start exploring these three areas rather than guessing what’s wrong.

Instrumentation vs. analytics: Measuring the user experience sometimes involves instrumenting dedicated sensors within your product or service (physical or digital) because different analytics packages don’t always give you the indicators you need.

In vs. out of context: When measuring attitude (what people think), it’s very easy to fall into the trap of sending a survey once in a while, yet a survey is not “close” to the experience. The best way would be to ask during or immediately after people have a certain experience. Leave surveys to questions that don’t make sense to ask otherwise.

Image for post
Image for post
At WeWork, we experimented with measuring attitude toward different parts of the experience. For example, we asked about satisfaction with phone booths and printers in those spaces, about events using an app we created, and left asking about the office chairs to a quarterly survey (chairs don’t change much and it didn’t make sense to put a happiness meter on each one).

🥇Only measure things you have full intent to change

🥈Reach team agreement and shared understanding

🥉Only measure things that are core to the experience

🎯 In a separate series of articles, I share a set of useful KEIs to measure based on the Google HEART framework:
Measuring user happiness
Measuring user engagement
Measuring user adoption
Measuring user retention
Measuring user task success

Meanwhile, please 👏👏👏, share, and comment. I’d ❤️ to hear from you. How difficult (or easy) is it for you and your team to decide what to measure?

If you liked this article, check out:

You can also find me on Twitter if you’d like to follow more of my research thinking!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store