Quick n’ Dirty Card Sorting for Designers
Complete a study in just a day
I strongly believe that oranges belong in a list of salad ingredients. I’ve had people agree with me, and others tell me that in no way, shape, or form does fruit belong in a salad.
Heavy stuff.
The way we categorize stuff in our heads is so unconscious we’re usually not aware of it. Yet its power over how we experience and perceive the world is incredible. On top of that, our “mental models” will differ depending on culture, language, upbringing, and so on.
Product designers are in the privileged position of understanding the models our users inherently have, and more importantly that those models likely differ from our own. Only after taking a walk in their shoes can we be confident that we’re structuring the experiences we build appropriately.
Luckily, a method of doing so has been on the market for a very long time: card sorting. It’s free, it’s proven, and plenty of others have written about why it’s valuable. It’s the information architecture research thing.
So you’re gonna go do it, right? This week… ok, you’ll ask next month at quarterly planning. Ok, maybe in a few months, when the developers have their “clean-up time” and there’s less pressure to feed the beast.
Finding the time or money (saying nothing of management-buy-in) to do research can be daunting, especially in smaller teams. I know.
But pucker up, cuz I’m gonna show you how to complete a quick, sneaky, study in just a day. Don’t even tell anyone you’re doing it.
Here’s how
If you’re still reading, I’ll assume you believe this is a valuable UX tool, regardless of whether your product has been around for 10 years or is still sketched in dried-out marker on a whiteboard.
Decide what to categorize. Over here, we were making some big changes to a music streaming platform. We wanted to validate our hypothesis that format (“live show”) and genre (“rock”) were the most useful categories for users to find a mix they might like. You will have your own reasons. Maybe you’re handling news articles for an online magazine and your boss thinks authors are better than topics as the main navigational links (eek!) Maybe it’s dresses in an online clothing store and the team can’t decide if occasion (“prom” or “bachelorette”) is an important filter. Either way, you’ve got something to categorize.
Recruit participants. So sneaky — for now, you’re gonna send this study to ten internal stakeholders involved in your project: your product owner, the other designers, the editors. It’s OK that they’re “too close to the product,” for now. So we can skip this step!
Set up your test. Were’ going to use OptimalSort (no affiliation) for now: it allows unlimited free studies of up to 30 cards and 10 participants each, and it has some very detailed analysis tools. Start a new study and add your 30 cards. For a news site, you might use an image-headline-author combo. For dresses, you might simply use an image of the product. During our test, we decided to use written teasers of the mixes as our cards. Even better, but not yet possible with the tool, would have been a short clip of the audio itself to represent our content most accurately.
Note: Since this is an “open” card sort that allows participants to create and label as many categories as they want, we need to ensure that we check the option “require all categories named.” Otherwise we’re left blind as to how they went about sorting.
Let the magic happen. You’ve got their email addresses already. Throw them in there and press “Launch Study.” People are opinionated, especially with things they work on eight hours per day. They’re also looking for distractions. I’m willing to bet you get at least 7 of 10 responses. Bonus: Sit and watch a few people as they do the test: you can ask them why they’re grouping certain cards together, giving you richer insights. Just be sure they know they’re not being tested, the product is.
Analyze the results. There are some useful visualizations to see what methods the participants used. Did author or topic come out on top? A tie? Something unexpected? Did occasion show up for dresses? Over here, we expected the format of the mixes (like “live show” and “chat”) to be popular, as well as the musical genre. But there were some surprises: by mood (“pure emotion”), by theme (“artist profile”) and by geography (“UK”).
Given this is a preliminary test the results won’t be statistically significant. They’re also biased since your participants were stakeholders. That’s OK — what we want is to stir the pot a bit, by showing that people have different methods of categorising information.
Have the talk
This part will last longer than the first day — and it should.
If the participants had varying methods by which to group the cards, then we have more proof that it’s a tough, subjective, process. Hopefully the exercise convinced your stakeholders (incl. yourself) of this. The next step is to perform a larger study with representative participants to find out what the most common main method is for them. That’s the good stuff.
If most participants seem to agree on one method (say, many used occasion to sort the dresses) then we’re still in a position where this hasn’t been validated by those who will actually use the product: users. There’s a good chance this consensus scenario will happen since our placeholder “participants” are so familiar with the topic that it’s impossible to avoid bias. Again — time to perform a larger study with real potential users.
Over here, we ran our quick n’ dirty test with ten members of our client’s internal team. As expected, this surfaced some differing opinions and got us super excited to see what real users might think! It also proved to be quite subjective, many cards having multiple categories they could be in. I dare to assume dresses will be easier.
Our next step was to set up a paid test with people on our beta list, create a short screener so we could later filter out non-representative users in the results, add a few more cards, make some tweaks to the instructions, pay for a one-time study ($99), and let the chains off.
We’re analyzing those results now, but it’s safe to say we’ll be making some adaptations to the product. It was quite enlightening, especially some of the comments participants made.
Good luck
I’ve included everything to get you going on a quick test above, but there are a few more things to keep in mind when setting up your later, larger, card sorting study: like when to leverage open or closed sorting, or how to avoid priming participants towards certain categories with trigger words.
I hope some interesting conversations pop up as a result of your initial card sorting round. Your efforts will reap some real real benefits for users. And that’s what we’re looking for.
Learn more about what we do over at acolorbright.com or follow us on Twitter.
