How to measure design (Part 1/3)
A reflection about design ROI, maturity and value.
Over the past 10 years I've been working as a design leader in many different industries (Fintech, Edtech, Insurtech, Ecommerce,…). Usually when the team I led reached a certain level of maturity some questions arise:
- What KPIs are accountable to designers?
- How to measure that we are evolving in design quality?
- How to evaluate designers?
- How to measure the impact of design?
- How to prove the return of investment in design?
I confess these are not "easy to answer" questions. Some years ago I wrote 2 articles about this subject with my main findings:
At this first article, 6 years ago, I shared some thoughts around how to measure design processes, design competencies and design results using HEART framework.
Some years later I dove into measuring results, thinking broader about outcomes of design to end-users, based on a case I've worked about measuring user happiness and frustrations.
Since then, this topic has been dormant in my mind, I've worked intensely into many other challenges. And even though I always had to prove design over an over again, I never took the chance to write about it. So here it is: Finally I decided to summarize some learned lessons on this topic from the last years, specially during my experience as Director of Product Ops.
Define why you want to measure
The first lesson learned is about understanding why you want to measure something. It's very important to address the pains you want to solve.
- Is it headcount pressure/competition?
- Is it about ego disputes across the company?
- Is it genuine intension to invest in user centricity?
- Is it performance evaluation of designers?
- Is it a pitch to be more relevant to the board?
- Is it about connecting design to business metrics?
- Is it to better address educational budget and raise the level of design?
Each one of these situations defines different ways of measuring design. To be more didactic, I would like to propose a split into 3 major intensions: measure user experience quality, measure return of investment of design, measure design maturity.
1) Measure user experience quality
"If you don't take care of your customers, someone else will."
— Edgar Mitchell
Many companies starts their businesses full of purpose: they deliver value with a small team, it catches attention of investments, there is an ambitious future, let’s grow.
Suddenly they are shipping a lot, and someone ask “Does it worth it?”, someone else checks. No traction. Overtime, the legacy & debts start to grow, all MVPs shipped are incomplete, features engagement is low, so let's invest in marketing, throw tons of customers in a leaking bucket funnel. The classic Startup Death Cycle.
You worked hard and shipped many innovations in production, you launched a lot last months, your callcenter is blowing. You cannot hire more agents if not growing first your profitability. You cannot grow revenue if you do not offer support quality. You cannot grow profitable if hire more support agents. How to scale my business? Then comes consultants, investors recommended. With hope, we check best-selling books, reports arrive, wishlist of new features, you code it, you failed.
"Perfection is achieved, not when there is nothing more
to add, but when there is nothing left to take away."
— Antoine de Saint-Exupéry
Better than solving things right, is solving the right things. Even if you code with high quality, you have to first code it relevant. The Product-led approach focus on shipping outcomes over outputs. So it's not about coded features, but shipped value, because people pay for value. People value time saving, empowering, advantageous innovations. Not necessarily the most profitable ones, nor the most impactful to your business & lowest effort costs, they don't even mind about the most scalable ones, most of times, they hate automated cheap solutions.
How to code precisely what is relevant to our addressable market?
We can explain the basic product-led approach like this:
> Explore situation
> Measure the problems
> Pick the most important problem
> Understand the problem
> Define success criteria
> Create some solving hypothesis
> Pick the most potential solution hypothesis
> Validate it with users
> Make it smallest possible
> Measure results
> Refine it
> Launch it
> Repeat
Great! If you follow this magic formula you have success, no. The way you measure yourself, defines your preferences and behaviors.
Imagine you are lost in the middle of the ocean with a small boat, which situation is better?
- Having a broken compass, showing wrong direction, but you don't know.
- Not having a compass.
Not having a compass will definitely push you to find alternative solutions to measure your direction. The broken compass can make you extremely confident in the wrong direction.
If product is used by people, why do we measure business instead of people? Money is a lag indicator (GMV (Gross Merchandise Value), MRR(Monthly Recurring Revenue), EBITDA (Earnings Before Interest, Taxes, Depreciation and Amortization), these are consequences of many systemic human decisions. Using lagging indicators can slow down your learning cycles. It can be too late to fix it or to react on it.
People value products that value people, "Narcissistic product sucks."
— Kendra Shimmell
There are many ways to understand customer satisfaction. These human centric metrics are some ways to measure human behavior/perceptions. Just to mention some methods:
CSAT (Customer Satisfaction Index)
Imagine CSAT as a thumbs-up or thumbs-down from your customers. It’s like asking them, “How did we do?” and they respond with a simple rating based on their overall satisfaction with their experience.
SUS (System Usability Scale)
SUS is like observing someone interact with your product or service. It’s about how easy and intuitive it is for them to use. Think of it as watching someone navigate through a city — the smoother the journey, the higher the SUS score.
NPS (Net Promoter Score)
NPS is like asking your customers, “How likely are you to recommend us to a friend?” It’s about gauging their loyalty and enthusiasm for your brand. Picture your customers as ambassadors spreading the word about your business.
CES (Customer Effort Score)
CES measures how much effort customers have to put into using your product or service. It’s like evaluating a recipe — the simpler and quicker it is to follow, the higher the CES score. Customers want solutions that are hassle-free and efficient.
Worthix (Worth Index)
Worthix is about understanding the value customers place on your offerings. It’s like assessing the importance of different ingredients in a meal — some elements may be crucial, while others are nice to have. Worthix helps you prioritize what matters most to your customers.
CLV (Customer Lifetime Value)
CLV is like looking at the long-term relationship with your customers. It’s about understanding their value over time — not just what they spend today, but how much they’re likely to contribute to your business in the future. Think of it as nurturing a plant to yield fruits for years to come.
CSS (Customer Service Satisfaction)
CSS focuses specifically on the customer service experience. It’s like evaluating the service at a restaurant — did the staff make you feel valued and attended to? CSS helps you measure how well your team is meeting the needs and expectations of your customers.
CHS (Customer Health Score)
CHS is like monitoring the well-being of your customers. It’s about assessing factors like engagement, usage patterns, and satisfaction to determine how “healthy” their relationship is with your brand. Think of it as checking vital signs to ensure your customers are thriving.
CCR (Customer Churn Rate)
CCR is like tracking how many customers are slipping through your fingers. It’s about measuring the rate at which customers are leaving your business. Just like plugging leaks in a boat, reducing churn is essential for maintaining a healthy customer base.
CR (Customer Review)
CR is about capturing the voices of your customers in their own words. It’s like reading letters from friends sharing their experiences. Customer reviews provide valuable insights and testimonials that can inform future improvements and attract new customers.
HEART Framework
The HEART framework is like a health check for your business, focusing on five key areas: Happiness, Engagement, Adoption, Retention, and Task Success. It’s about making sure your customers are happy, engaged, and sticking around, while also effectively achieving their goals with your product or service.
It's very important to acknowledge that these metrics can be manipulated and are not comparable across different companies, the way you apply each method can imply into BIASED metrics.
So, measuring user experience means measuring many human dimensions. The most important part is about what you do with these user centric insights and how you move the needle. Money would be a consequence of customer awareness, desirability, adoption and loyalty.
Designers can definitely help measuring and understanding these insights. They can also help you experimenting solutions, and they even can help you to code more relevant products. Last, but not least, not forgetting about the part that they can help you story tell it to create awareness, desirability, adoption and loyalty.
Let me stop here, if you reached this part, you are a warrior, comment this article and I promise I will answer with all my focus.
I will publish the next parts soon, so follow my profile to be notified later :)
NEXT ARTICLES:
How to measure design (Part 2/3)
Measure return of investment of design
"If you think good design is expensive, you should look at the cost of bad design."
— Dr. Half Speth
How to measure design (Part 3/3)
Measure design maturity
"Good design, when it’s done well, becomes invisible.
It’s only when it’s done poorly that we notice it."
— Jared Spool
