The HEART of the Matter: Google’s Framework for Measuring Design Value

Is design undervalued at your organization?

Even as more designers are welcomed at the proverbial table, there’s a lingering sense that executives and colleagues from other departments don’t understand design’s impact on the bottom line. But it doesn’t have to be this way.

The field of design has matured to the point that we have the tools, frameworks and expertise needed to successfully communicate to non-designers what we do and why it’s important.

I recently hosted a Fireside Chat with two of my favorite experts on the topic, and this post follows one of the threads from our conversation.

You Gotta Have Metrics

Not everything about user experience can be measured.

Data will always have limitations and flaws. But, when product, engineering, finance, sales and marketing are all tracking and talking in metrics, you’ve got to meet them where they’re at.

These groups will likely have specific analytics they focus on, like retention, churn or conversion. On the other hand, design teams — in considering the entire user experience — can strive to understand how all of these metrics fit together into the bigger picture. Ideally, part of the value they bring to the organization is a working knowledge of the different types of metrics in the toolbox, and when to use them.

Over the last decade, Google’s HEART has become a go-to framework for thinking about the purpose of different metrics. That’s why I was thrilled to have Kerry Rodden in this Fireside Chat. Before setting up shop as an independent consultant, she was a senior staff UX researcher at Google, and a co-creator of HEART.

Kerry originated the role of quantitative UX researcher at Google, analyzing large-scale usage data to answer product-design questions. That’s how she found herself at the heart of HEART.

How Google’s HEART Works

Designers regularly came to Kerry asking what metrics they should monitor on their dashboards.

Over time, she and her team realized they were taking designers through the same guidance over and over: What’s the goal of your project? What are you trying to achieve? How will you know if a redesign is better?

They boiled it down to the HEART acronym so designers and project managers could easily remember it and use it to lead conversations about what matters most and how to measure it:

H is for Happiness

There’s not an obvious metric for happiness. To understand if users are satisfied or delighted with a product experience typically requires some extra legwork in the form of interviews or surveys. Over time, you may be able to correlate this secondary data with primary usage data to recognize patterns of behavior that imply happiness.

E is for Engagement

Engagement is much easier to measure with primary data. Simply put, it’s how much and/or how often a person uses a product or feature. Engagement is particularly telling in consumer applications. It’s less useful in enterprise settings where a user may have no choice in the applications he or she uses.

A is for Adoption

How many new users does a product or feature attract over a period of time? That’s an adoption metric. You could make the argument that short-term adoption metrics reflect the success of marketing efforts. But over the mid- to long-term, the user experience should drive adoption via reviews and word of mouth.

R is for Retention

What percentage of signups become regular users? What’s the churn rate? These are metrics of retention and they are most valuable when analyzed across discrete time periods within the user journey. This allows you to zoom in on friction that may occur in particular stages.

T is for Task Completion

Usability testing and benchmarking analytics are two methods for capturing task-completion metrics. What’s often measured is the amount of time spent on a particular task. Other common metrics include the percentage of users who complete a task, such as creating a profile or checking out.

But Wait, There’s Another Axis

If the HEART acronym is the vertical axis, it’s the horizontal dimension of the framework that helps you know what to do: GSM.

It stands for goals, signals and metrics.

The idea is that for a characteristic such as Adoption, you would determine the relevant project goal. Then determine the data signals that reflect user behavior toward or away from the goal. Then define the parameters of the metric you will track to reflect progress.

According to Kerry, designers are sometimes tempted to fill in HEART metrics on their own and think they’re done. “That’s completely not the intention,” she said. “The idea is as with anything, an iterative process is going to be what gets you there, because you can’t just make assumptions about what that particular signal means if you see it in your logs.”

She stresses the importance of getting out and observing users — making sure the data signals represent what you think they do. “The combination of what is going on and why it’s happening is the most powerful,” she said. Without the why, your metrics won’t have the credibility they need.

More Frameworks and Resources

Google HEART is just one of several useful metric frameworks. Leanne Waldal was the other guest in our Fireside Chat. For the past several years, Leanne has led research at Dropbox, Autodesk and New Relic, and she has a thoughtful approach to measuring and communicating the value of design. She highlighted several other frameworks and resources you may want to explore:

• SUS — The system usability scale (SUS) is a ten-item questionnaire developed by John Brooke in 1986 to measure the perception of system effectiveness, efficiency and satisfaction.
• UMUX — The Usability Metric for User Experience (UMUX) is similar to SUS, but shorter with only four questions (two positive and two negative).
• UMUX-Lite — Same as above, but boiled down to just two items graded on a seven-point scale: [This system’s] capabilities meet my requirements; and, [This system] is easy to use.
• NPS — Net Promoter Score (NPS) is a satisfaction survey used to gauge customer loyalty. Caveat emptor, take this one with a grain of salt.
• NVS — Developed by B2B International, Net Value Score (NVS) aims to reflect perceived value of a company in a particular market.
• HappyOrNot Ltd. — This Finnish company makes terminals that measure customer satisfaction via smiley and frowny faces.
• MeasuringU — The blog produced by these measurement specialists contains a wealth of information.

This post provides a high-level perspective on the field of design metrics, and on first read it might feel a bit overwhelming. But know that the aim is not to measure everything you can — far from it. Instead, determine the one or two most important things to measure and start there. With this approach you’ll gradually build your skill and understanding of design metrics and how to bring them to the table effectively within your organization.

To dive deeper into the subject, check out this and other webinars hosted by InVision.

Plus, in a separate article, I’ll share Leanne and Kerry’s best tips for communicating the value of design through metrics.