Balancing customer and business value when measuring the ROI of UX Design
By far, the most common challenge that user experience practitioners grapple with is “How do I show product leaders and other decision makers that building feature x is important to the business?” This question is almost always followed by “How can I measure the impact of the improved design if I can’t get support to improve the design?”
Why is measuring potential UX value difficult?
Measuring the potential value of anything is difficult, and this is multiplied tenfold when it comes to user experience. Unlike our sales and marketing colleagues, it is not always possible to draw a straight line between our UX design work and the dollar values that executive stakeholders care about. When we are unable to show the potential value, the business is at risk of being the focus of headlines like this-
All too often, we have to depend on secondary sources and the imagination of our executive stakeholders to support statements like “The improved experience will make customers happier, more loyal, and as a result, more profitable.” This statement is by no means wrong, and executive stakeholders in a digitally native organisation will happily connect the dots. However, the sales function will typically win more support and budget because they are able to confidently say “if we are able to contact 10,000 more customers this quarter, we will double our profit.”
If we don’t give leadership numbers, it is difficult for them to see the progress and improvements we’re making as a team, or how we are helping to progress the organisation as a whole. It is important for us as UXers to be able to speak the language of our executive stakeholders if we want to craft better experiences, and this begins with getting better at measuring potential value.
How to start thinking about measuring value in UX?
This is a bigger topic that I will cover in another article, but for the purposes of this article, I like to think about measuring experience as a marathon.
Crossing the finish line is the objective- like the big, inspirational organisational goals that are represented in measurement frameworks like OKRs (objectives and key results) or BPMs (Big picture, priorities and measures).
The milestones we cross on the way to finish line e.g. first 1km, first 5km, etc, are the key results or KPIs that show our progress to the finish line.
All of these metrics are relevant to measuring UX value, but potential value is measured with a type of KPI called the Problem-value Metric. This metric is a key result that represents obstacles that we have to overcome on the marathon course, like the ‘hill of pain’. The problem- value metric tracks how much that obstacle is costing us during the marathon e.g. added time, reduced endurance, knee wear and tear, or in my case, certain injury.
What is a Problem- value metric?
A Problem- Value Metric highlights customer and business value by showing the cost of poor UX. Jared Spool explains that a problem value metric answers two simple questions-
1. ‘What obstacles are preventing our users from achieving their goals?’
2. ‘How much are those obstacles costing our organization?’
Whenever we ship a sub-par product experience, our organisation incurs a debt that it will eventually pay, through wasted development effort, lost productivity, lost sales, and increased support.
The problem value metric connects UX KPIs to business KPIs. It is a number that tells us how big are the obstacles to achieving a great user experience, and how much those obstacles are costing our organisation every month, quarter, or year.
Providing monetary costs allows executive stakeholders to do a like for like comparison of UX to other parts of the organisation like sales, marketing and operations. This number value also shifts executive mindsets from viewing designers as people that makes things pretty to a team that saves the organisation millions of pounds.
How to identify the right Problem- value metrics
To identify the right problem value metric we need to measure the current experience, and deeply understand the problems that prevent users from achieving their goals. The more we fall in love with the problem, the more we quantify it, the more likely we can get to solutions that actually fix the problem. These are the steps to get you on the way to identifying the right problem value metric:
- Map the current experience. First, Map out the current experience on a journey map with milestones that users have to accomplish to have the experience.
- Identify pain points or moments of frustration. Map each milestone on a scale of delight to frustration.
- Identify the cost of frustration. Whenever we deliver a poor experience, it creates UX debt that, over time, accumulate to substantial amounts in the organization’s bottom line.
These cost(s) of frustration are observable across one or more of these drivers:
Lost sales. Sales are lost when customers find the product confusing, complex, or harder to use than a competition.
Increased support costs. A complex and confusing product often leads to more support and training costs. Labour and support costs are often among the biggest cost centers for product organisations, and is therefore a great starting point for defining problem value metrics.
Decreased productivity. Employees are less productive and motivated when they spend more time dealing with product complains and bug fixes.
Increased developer waste. Building features that no-one uses is more or less a wasted development effort.
Increased developer costs. Going back to fix features that have already been deployed requires navigating various dependencies that increase the complexity, time and cost of rework.
4. Calculate the cost of frustration. We have to do a bit of detective work to identify which moments of frustration have the biggest impact on the drivers above. The cost drivers that are impacted the most by moments of frustration are good candidates for problem-value metrics. Usually, the right metric for each moment of frustration is the one driver that stands out by sheer magnitude, or because it is especially relevant to the context of the business.
Once we identify the right metric, we need to constantly track it and calculate costs over time. The more we track and report these costs, the more attention we will draw to the size and importance of the problem. This is an important step so I’ll explain the calculation with a practical example.
How to calculate the Problem- value metric
Let’s step back to the questions that led down this road and add some more context. Let’s say you’re working on a retail banking app, and after doing some primary research to map out the current experience you identified that the password reset step was particularly frustrating for users.
You take this away to your cave of perpetual ideation and have 3 alternative workflows that you strongly believe will enhance the experience. You are now at the stage where I started this article, and you’re asking yourself-
“How do I show my colleagues that building a new password reset workflow is valuable to the bank ?”
“How can I measure the impact of the improved password reset workflow if I can’t get support to improve the design?”
After reading this article, you know that this frustration will be reflected either in increased churn, support costs, developer costs, developer waste, or decreased productivity. You also begin looking into support costs since you know this is often a large cost driver in product organisations.
You go to your organisation’s trusty Zendesk support dashboard and filter all the password reset tickets. You see that all the password reset issues require a customer support agent to intervene and help the customer. To calculate the problem value metric for password support re-design, you need to calculate the cost per password reset ticket as follows-
🧮 (total cost of operating support ➗ total support tickets) ✖️ total password reset requests
To get to the figure for a total cost of operating support, you can speak with your head of support, or perform some back of the envelope calculation as follows:
Number of customer support reps ✖️ Estimated customer rep. salary and employment costs
Back-of-the-envelope calculations are better than nothing since an imprecise number can still more or less accurately express the magnitude of the password reset problem. We care about the magnitude, not precision.
💡 Tip- If you want to improve the accuracy of your rough estimation then identify someone you think might have a better idea of the cost, and mention that you have a number but you’re not sure if it’s accurate. More often than not, this person will go off an do some quick research and get back to you with a more accurate number.
Amplifying the impact of Problem-value metrics
- It’s also important to point out to your stakeholders that the problem value costs might keep adding up if nothing changes. It’s not just about how much the problem costs this year, but it’s also about how much was spent on the problem in previous years, and how much the organisation will continue to spend in the next 2,3….10 years.
- Fixing the problem is a one-time cost that goes away. Even if fixing it seems expensive compared to ongoing costs, the ongoing costs will continue to increase. We can therefore use the problem-value metric we calculated to make projections about what it would cost if we don’t take action and projections for what it costs if we did take action. The value of solving this problem is clear when positioned this way. The return on investment is clear.
- Compared to many other ROI cases, solving these problems will not require a new investment. For example, with the support costs associated with password reset, we are simply taking existing investment into support operations and re-directing it into a feature refinement.
When we use and continuously track problem-value metrics as part of our UX work, we are able to show others the size of the problems we are solving, stack the costs of our work up against other functional costs, clearly show the return on investment of UX design, and make projections about costs if we did take action versus what it would cost if we don’t take action.
