Chef Antonella opened her new restaurant six months ago. Everything has been going really well, it seemed. Revenue is up and to the right, tables are reserved in advance, and the restaurant is full and lively every single night. Business metrics show a very positive picture.
Chef Antonella also measures two key experience indicators she cares about. These paint a different picture. Close to 100% of customers are new. She has almost no returning customers. Second, on average, 70% of food is left on the plates. Something is wrong although business is great.
🔑 Why measure experience?
Key Experience Indicators (KEIs) provide a quantitative score of a specific, important, and actionable phenomenon related to using a product or service. Measuring KEIs has the following benefits:
- KEIs provide information to decision makers.
- KEIs precede or predict business outcomes.
- KEIs give insights into qualitative findings and customer anecdotes.
🔬Considerations of KEI measurements
What vs. why: KEIs will never explain themselves. To understand a KEI, one must invest in qualitative research.
Good vs. bad: It is not always clear whether KEIs are showing a positive or negative phenomenon.
Can you tell if these are good or bad?
- Average time to first click is 49 seconds.
- Average time on site is 27:38 minutes.
- Homepage bounce rate is 99%.
Attitude vs. behavior: what people say does not necessarily match how they behave. People’s attitude is interesting yet behavior is much more telling. Give more attention and weight to behavioral rather than attitudinal metrics.
Chef Antonella’s restaurant is providing great coffee. Yet if you ask one of Antonella’s customers to rate her satisfaction with coffee immediately after she found a mistake on the check, coffee might be getting low scores. Coffee consumption, however, measured in the same exact time, might tell a more realistic story.
Overall vs. specific: Measuring specific phenomena gives you the ability to focus efforts and save resources. When you measure NPS or overall satisfaction, you get scores that do not help you understand what goes well and what needs fixing. If you get a score of 66 for your product’s NPS, do you know why you lost 34 points?
Alternatively, when you measure a lot of smaller experiences, for example, at a hotel, you may find that while your overall UX score is 93%, you lost 7 points because coffee, room cleanliness, and the online checkout process got low scores. You wouldn’t know why and yet you can start exploring these three areas rather than guessing what’s wrong.
Instrumentation vs. analytics: Measuring the user experience sometimes involves instrumenting dedicated sensors within your product or service (physical or digital) because different analytics packages don’t always give you the indicators you need.
In vs. out of context: When measuring attitude (what people think), it’s very easy to fall into the trap of sending a survey once in a while, yet a survey is not “close” to the experience. The best way would be to ask during or immediately after people have a certain experience. Leave surveys to questions that don’t make sense to ask otherwise.
🥇Only measure things you have full intent to change
Spoiler: you can change anything in the experience you created. It’s all a matter of motivation to do so. This one is important. Your customers will stop cooperating with your attitudinal measurements and get angry with you if they notice you don’t change the experience after they provide feedback.
Test question for the team: Will we take action once we have data for this metric?
🥈Reach team agreement and shared understanding
Agreement within the team and senior stakeholders that a certain KEI is important. Disagreement results in ignoring KEIs and not taking action. Make sure to create a shared understanding and acceptance of what’s being measured.
Test question for the team: Do we agree this metric is important to measure?
🥉Only measure things that are core to the experience
Measuring everything is measuring nothing. Focus on measuring things that are truly central to the experience. Together with your team, reach agreement on what these are. You may have some debates here but for the most part, those core things are going to be easy to identify.
Test question for the team: Does this metric measure a thing that is core to the experience?
🎯 In a separate series of articles, I share a set of useful KEIs to measure based on the Google HEART framework:
Measuring user happiness
Measuring user engagement
Measuring user adoption
Measuring user retention
Measuring user task success
Meanwhile, please 👏👏👏, share, and comment. I’d ❤️ to hear from you. How difficult (or easy) is it for you and your team to decide what to measure?
If you liked this article, check out:
You can also find me on Twitter if you’d like to follow more of my research thinking!