Make card sorting even more impactful with this simple scoring formula

Often times designers struggle with their jobs because their work is subjective. Stakeholders, Product people, or even other Designers might prefer a particular experience because it just slaps (for them).

Doing user research is one way to move away from this. We can start to back up our design decisions with evidence, strengthening it when sharing with others.

A simple exercise during User Research is card sorting. Here’s a quick rundown:

1. You run interviews with users on their workflow, their pain points
2. Post-interviews, you cluster the common pain points
3. You list out some solutions to pain points
4. You ask the interviewee to rank these in terms of importance
5. They can bucket into 3 sections, Must-have, Nice-to-have and Can-live-without
6. You force interviewees to only put a few into Must-have, to really get to the most important ones

Card sorting: a powerful, simple research method

While this exercise is simple, it will likely sit alongside a broader piece of User Research, which as a whole isn’t something that is simple & easy to digest. It has depth and therefore requires a proper review of the findings.

This can lead to a problem in Product teams, however. When presenting a case to stakeholders with some context on the projects in progress, they may listen closely to what you have to say. Senior stakeholders or Management, however, can be a different story.

Their way of communicating is different.

These are people that want more punchy outputs, like numbers.

So instead of having a situation where you say X, Y, and Z are ‘Must-haves’ for the product, which can introduce some subjectivity due to the vagueness of something being a must-have, you can start to quantify and present a ‘Score’.

Essentially we’re going to try and move away from words into numbers, to speak the language of the business and senior stakeholders, to better convince them of our findings, and drive the most impactful improvements for the product.

For context, this is my process for doing card-sorting with users remotely:

1. Create & share a Google Form asking users to rank solutions (Figure 1)
2. Export these to a Google Sheets file
3. Count the rankings for each solution
4. Score the 3 ranks differently
5. Sum up the individual scores into an aggregate
6. Rank the scores

Figure 1. Google Form

Let’s run through how the scoring works.

Scoring methodology

I give the 3 rankings these 3 scores to create a bigger spread in aggregate scores to highlight differences more easily across solutions and to prioritize/skew for solutions that we deemed nice or must-have. If users think they can live without something, we can get to it much later. These are the weightings for the 3 rankings:

Must-have: 1
Nice-to-have: 0.5
Can-live-without: -0.1

Once the data in the form (Figure 1) has been exported, we can start to create the aggregate scores for the solutions inside Google Sheets.

The formula counts the number of responses for the 3 rankings, then those are weighted and added up to create an aggregate score.

This is how the whole table looks at the end:

After the number crunching

Once we have all this data nicely laid out and connected, we can then start to communicate this with our stakeholders.

If we take the example above, the most impactful solution was ‘Submit building evidence inside ARCA’.

Previously, we might have communicated this to stakeholders by saying — Submit building evidence was the most popular must-have solution that users chose.

Doesn’t really have a ring to it does it?

Here’s what we’re going to say instead:

‘During User Research, ‘Submit building evidence in ARCA’ had an Impact Score of 75%, and was the top-ranked solution chosen by users’

BOOM.

If you want to add a sparkle of storytelling, you could show the 75% on the screen and create intrigue. This is one of the tips I shared on enhancing your presentation:

https://www.instagram.com/reel/CeO7-acjH9A/?utm_source=ig_web_copy_link

We’ve taken a wordy sentence and brought a digestible and punchy ‘Impact Score’ to it.

All the solutions with their Impact Score on the right.

Storytelling is powerful, and this is exactly how you should use storytelling to better communicate your user research.

Sakky B
Co-founder, ZeroToDesign