Dialing in the Detail: Getting the Most Out of Your Open Card Sort

by Vesna Gronosky, UX Researcher @ Chewy

Open card sorts are great when planning a new website, or when redesigning the one you already have.

In principle, it’s easy to do:

1. Show the participant 45–60 cards with generalized items.

2. Ask them to group these items into whatever clusters make sense

3. Ask them to name the clusters

From the results, you learn how users think about your content, how they structure it, and what they call the categories they create.

But have you ever hoped to get a site structure with detailed sub-categories from a card sort, and gotten Dog Stuff, Cat Stuff, and Other Pet Stuff instead?

It happens. It happens because getting the right level of detail from a card sort is harder than it seems.

How do you fix it?

First, decide the level of detail you’re looking for. Are you hoping to create top level categories? Detailed subcategories? Both?

It is possible to ask too much of a single round of card sorting. When looking for detail, a phased approach can yield better results. At Chewy, we use the phased approach to dial in the level of detail needed for the results we’re looking for. And we have found that choosing this approach saves us time in the end.

The Phased Approach

For big sites with diverse content, it can be best to split the card sort into two phases:

Phase 1: Derive top level classification

Phase 2: Run separate card sorts to determine sub-categories for each top level category

Otherwise, there is a risk of being too vague. Asking participants to cluster and name 45–60 Dog items will yield a vastly different type of structure than asking them to cluster and name 45–60 Dog, Cat, Hamster, Bird, and Fish items. For top level categories, the Dog, Cat, Hamster, Bird clusters are the correct level of specificity. For Dog sub-categories, 45–60 Dog items will likely yield Dog Food, Grooming, Dog Apparel, and similar more specific categories.

Both of these approaches have their place, but it is important to pick the right one for the right purpose. It is not always possible to answer both questions in one round.

But what if I increase the number of items?

Theoretically, participants can build a complex hierarchy when given a larger number of cards. 100 or 200 items force complex classification. But chances are, participants would get tired, careless, and abandon an unmoderated remote study. Especially if the items they are being asked to classify are complex and require expertise. Pharmaceuticals, for example, require quite a bit of time and thinking to sort meaningfully. Asking participants to classify more than 60 pharmaceutical items, complex business terms, or pieces scientific equipment is likely to result in sloppy classification or study abandonment.

Things to Consider:

Doing a card sort in phases takes more time. The first phase determines top level categories. The second phase usually involves several category card sorts. Fortunately, category sorts can be performed simultaneously if done as remote unmoderated studies.

Analysis tends to take longer because each card sort has to be analyzed separately, followed by a meta-analysis step to pull it all together. But the result is a navigation with the appropriate level of detail.

Lastly, it is important to tree test the finished navigation. The card sort shows how participants cluster items when left to their own devices. Whether they can easily retrieve items from the same structure is a separate question.

Conclusion:

Card sorting a great way to find out how users structure and label content, and it’s an excellent starting point for a conversation that puts the user at the center of the information design. But because card sorting usually happens as part of a larger design effort, time can be scarce.

Dialing in the detail saves time and rework. Decide whether you want to create top level categories, detailed sub-categories, or both.

1. For top-level categories, select items across categories

2. For sub-categories, select items from within the category

3. When top-level categories and sub-categories are uncertain, consider a phased approach

With unmoderated remote sorting, the second phase can be sped up considerably by running multiple sub-category sorts at the same time. Getting it right does not have to take a long time, and the results will be worth the effort.

by Vesna Gronosky

UX Researcher @ Chewy

If you have any questions about careers at Chewy, please visit https://www.chewy.com/jobs