Guide to Conducting Card Sorting Analysis

Imagine you’re designing a new online store. You have many ideas for categorizing products, but how do you know which makes the most sense to your customers? This is where card sorting comes in. It’s like a detective game for understanding how people organize things in their minds.

By watching participants sort information into groups, you get a glimpse into their thought process — how they naturally categorize the world around them. This secret knowledge is the key to crafting an information architecture (IA) that feels intuitive and effortless for your users to navigate.

Read more on how to conduct card sorting analysis?

Sorting Methods: Open vs. Closed Card Sort

The analysis approach hinges on the type of card sorting you conducted:

Open Card Sorting: Participants create categories for the cards, revealing their mental models for information organization.

Closed Card Sorting: You provide pre-defined categories, and participants sort the cards into those categories. This helps assess how well your existing IA aligns with user expectations.

Open and closed card sorting influence the focus of your investigation rather than the exact procedure. Open card sorting analysis guides you through creating an information architecture from scratch, while closed card sorting validates — or disproves — the existing information architecture.

With closed card sorting, pay attention to how users organize concepts and if this aligns (or diverges) from your existing IA. If particular cards are constantly placed in several categories, it shows they are confusing to users and might need separation into multiple topics.

Steps to Conduct Card Sorting Analysis

Card sorting analysis is a valuable technique for understanding how users categorize information, which can inform the structure of a website or app. Here are four steps to conduct card sorting analysis effectively:

Step 1: Take a Big-Picture Look

Start by reviewing your results and looking for intriguing trends or standout groupings. Rather than becoming too technical, look for more significant implications for user experience.

Look for the most often-created categories in your participant database for open card sorts. You can build your information architecture on these categories. For closed card sorts, compare your outcomes with existing categories and look for any standout trends, such as cards frequently categorized differently from your website’s categories or unused categories.

Other recurring themes and patterns could be:

Normative clusters: Consistent arrangements indicate a shared mental picture among participants.

Overused categories: Too many cards in one category suggest a need for a more focused subcategory.

Combining categories: Recurring analysis in several categories indicates these concepts should be accessible from different points in your IA.

Step 2: Clean and Standardize Your Data

To advance your information architecture, compile all your data into one location and ensure it is flawless before further examination. Convert physical card sort results to a digital format, often using spreadsheets or a dedicated card-sorting application. Standardize users’ assigned categories if you used open or hybrid card sorting.

For instance, similar labels like “about us” and “about the firm” can be combined into a single category. This standardization allows for a more accurate and streamlined analysis.

Step 3: Examine Your Card Sorting by Hand

Examining your card sort analysis manually can be meticulous but insightful. Approach this step with both quantitative and qualitative methods:

Quantitative Analysis:

Clustering: Group analysis results together, showing clusters based on frequency.

Dendrograms: Tree-like diagrams that show hierarchical connections among data points.

Matrix Spreadsheets: Determine which categories cards were sorted into most often.

Qualitative Analysis:

It provides in-depth insights and can be more time-consuming, but it does not require statistical generalizations, allowing for smaller sample sizes.

Step 4: Analyze Agreement Scores

Analyzing agreement scores helps quantify how consistently participants group the same items.

Calculate Agreement Scores: Use a card sorting analysis tool to measure how often participants grouped the same cards.

Identify High and Low Agreement Areas: Focus on high agreement areas as a strong foundation for your IA and analyze low agreement areas to understand participants’ differing interpretations.

Consider the Context: High agreement scores highlight intuitive groupings, while low scores may indicate ambiguous cards or categories needing revision.

Use Heatmaps: Visualize agreement scores to identify consensus areas needing more attention quickly.

Step 5: Share Your Findings with Stakeholders

Create a UX research report to share your findings. Include:

a. Your chosen card sort method and its relevance to your research plan and objectives

b. Demographics of the testers and participant acquisition methods

c. Detailed results, discoveries, and insights

d. Solutions based on your insights and their impact on your UX roadmap and research plan

Conclusion

You can transform raw card sorting data into actionable insights by employing these analytical steps and considering the “why” behind user behavior. Card sort analysis empowers you to create an information architecture that aligns with user mental models, resulting in a website or application that is intuitive, navigable, and, ultimately, a delight to use.