Measuring user happiness
This is part 1 in a series of articles about measuring Key Experience Indicators (KEIs). In this series I go deeper into the Google HEART framework for large-scale data analysis. The framework was put in place to help choose and define appropriate metrics that reflect both the quality of user experience and the goals of your product. Each article in the series discusses one of the HEART dimensions — Happiness, Engagement, Adoption, Retention, and Task success. Enjoy and use it!
What is happiness?
In the context of products and services, happiness is a pleasurable or satisfying user experience. Additional adjectives to describe happiness are contentment, joy, and delight.
User happiness is a self-reported measurement, which means you have to ask people to rate their happiness rather than tracking their behavior. That also means that emotions and bias come to play here. For example, if you ask me about my satisfaction with the speed of my phone the day I learn that my carrier has surprisingly and significantly increased the price of my plan, I am probably going to give you a bad rating for the wrong reason.
Why measure happiness?
Understanding people’s attitude toward a thing (product, service, feature, process, etc.) is extremely helpful in identifying strength and weakness areas of that thing. In addition, happiness measurements can help understand the effect of a change that was implemented in the thing (redesign, introduction of a new feature, or removal of functions) after it was introduced and over time.
Key mistakes in measuring happiness
Measuring overall happiness with a product or service is not going to be very useful and actionable. When you do that, you are able to understand if you have a good or bad score, and you’ll be able to compare your score to the score of other products. That said, you will have no idea what caused the score to be imperfect, or go down.
Measuring happiness out of context is going to mislead and take you off rail. Happiness with a thing must be measured at the right time, the right place, and with the right person. If you ask about satisfaction using an annual survey while people use the thing once a day, people don’t remember what happened. If you measure satisfaction with a thing not during or immediately after people use it, you are measuring out of context. If you ask a doctor about satisfaction with a calendar that an admin is actually using, you are measuring out of context with the wrong person.
Over complicating things is always going to work against you. People don’t want to take surveys. Nobody enjoys responding to a 5, 10, 20, 30, or 40-minute survey. Even if it “only” happens once a year. Even if you raffle an iPad. Surveys should be quick and effortless to answer. How does a one-question survey sound? Make surveys fun and easy to respond to. You’ll get more, less biased responses.
Another over-complication comes in the form of huge scales for happiness questions. A scale has a number of options for an answer and there’s a lot of scientific research to prove that people interpret different points on the scale very subjectively. Thumbs up, thumbs down is simple. 3-point scales are simple. If you feel strongly about a 4 or 5-points scale, go for it. Not more. If you measure very specific experiences, it is very easy for a user to decide if they are happy or not.
Three happiness metrics
Actual NPS: Rather than asking your users to predict their future behavior using a NPS question and bizarre calculations, ask them about actual behavior they have demonstrated recently or even better, track it. Examples:
- “In the last two weeks, did you recommend us to a friend or colleague?” [Yes/No]. The score would be the percent of people who answered Yes.
- Monitor social media for promoter and detractor posts and score them based on their strength and direction.
Percentage of satisfied users: The percentage of users who indicated they are happy, satisfied, or delighted with a feature, product, or service. For example, if you have 100 users who responded to your question, 17 of them were unhappy, 24 were undecided, and 59 indicated they are happy, then your score is 59% (100–17–24=59). Your goal would be to increase this number.
Mean satisfaction score: The average satisfaction score considering all ratings. Continuing the example, your mean satisfaction score for your 100 users would be calculated this way: a happy rating gets 1 point, an undecided rating gets 1/2 point, and an unhappy rating gets zero points. Aggregating the points (17x0+24x0.5+59x1) gives you an average satisfaction score of 71%. Tip: Please please please be more sophisticated and calculate basic descriptive statistics to figure out the confidence interval. That would give you a more accurate description of your score.
Taking action on happiness data
User happiness data is not just interesting, it is actionable. Here are some actions you can take with happiness data:
- Set happiness goals for product teams. Challenge teams to get better scores for user happiness by setting goals. Set goals for different geographies: People coming from different cultures tend to complain more and be less satisfied compared to other ‘happier’ cultures. Set goals for products in different stages (‘Mature features’ goal is 90% satisfaction, new features’ is 70% in the first 6 months’).
- Identify problem areas and improve them. Configure your surveying system so that each time a customer indicates they’re dissatisfied, there’s an notification, and action is being taken. Maybe someone senior at the company contacts them, maybe something else. Use happiness tracking to turn frown faces into happy ones.
- Evaluate the success of a change. For example, set goals for redesigns (‘Get to 85% happy users 30 days after launch’).
In summary, happiness is an interesting, actionable, yet slightly biased metric to measure. It would not be a good idea to only measure happiness, yet if collected rigorously and combined with behavioral metrics, happiness provides extremely useful insights.