Card Sort for Information Architecture Redesign

What is a Card Sort?

Card sorting is a popular technique (generative method) that can help you gain insights into how your users/site visitors think about the organization of your online content; it helps you understand their mental model. This research method can be conducted in-person (offline) or using an online tool. My colleague, Andrew Schall, our Director of User Research, wrote an article on the pros/cons of these two different data collection methods and when to use them.

In addition to the decision to collect data either online or offline, there are two primary types of card sorting approaches, Open and Closed.

What is an Open Card Sort?

With an open card sort, there is no predefined structure. Participants are supplied with a list of items (content labels) and directed first to sort them into groups/categories. Participants are given the autonomy to group as they deem most appropriate. They are also tasked with naming / labeling the categories they have created.

The primary reason why an open card sort is chosen is to learn how users group content and the category names they deem most appropriate.

What is a Closed Card Sort?

With a closed card sort the structure is predefined. While participants are still given a list of items to sort, instead of creating their own categories they are directed to sort the items into the categories you have created. Their task is to place the cards into the category they think is the best match. While this is still considered a generative method, it also evaluates / validates the hypothesis that the categories provided are the correct ones.

Closed card sorts help understand how content is sorted when given a predefined set of categories.

What Else Do I Need to Know?

Now that you have a basic understanding of what a card sort is and the two options you have, you may be wondering how many participants you need to get reliable and actionable insights. As with many things in life, it depends. There is research that has been conducted by Dr. Tom Tullis and Larry Wood that is widely accepted as a baseline for how many participants to select. Spoiler alert, they recommend between 20-30. We have taken it a step further and have a recommendation that differs depending on the type of card sort study you are conducting.

When conducting an open card sort, we agree with Tullis and recommend 30 participants. Since this method has the potential for greater variability than evaluative testing, we recommend being more thorough in your approach. For reference, a sample of 30 participants gives you a correlation of 0.95 between the results from the test and the ultimate results.

For a closed card sort, we recommend a sampling of 15 participants. Again a closed card sort is generative in nature but the results are limited a degree by the predefined categories you provide. A sample of 15 participants will give you a correlation of 0.90 which we have found to be very acceptable in practice.

Do You Have Any Tips?

Through trial and error, we have identified some practical tips you should incorporate to make your card sort study as successful as possible.

1. Be mindful of participant fatigue and limit the number of cards you ask participants to sort. We recommend 30-40 cards max for an open card sort, especially online. This will yield a study that takes on average 20 minutes.
   
   
2. Eliminate order bias. Be sure to randomize the order of the cards and categories in your study.
   
   
3. Have the “right” users participate in the card sort study. The participants in this case should NOT be your designers, developers, VPs, etc. but rather representative of your actual site visitors.
   
   
4. “Clean” your data before beginning your analysis. This is especially true for online card sorts. You want to ensure that each participant took a “good faith effort” when completing your study. Something simple that you can do is place cheater cards or red herrings that instruct users to sort the card (e.g., Place this card in the XYZ category). This provides a quick single place to check to validate if users are reading the item labels and trying their best.
   
   

How Have You used Card Sorts in Practice?

One of our more robust approaches to identifying the appropriate IA for a site was conducted for a financial services company. They were looking to provide their site visitors with an easier way to locate helpful financial advice articles/content from their website. Their analytics showed that the structure they were using was ineffective. They were so dissatisfied they wanted to begin anew.

We recommended an approach that included:

1. An open card sort to identify how consumers organized this type of information.
   
   
2. We followed it with a closed card sort that used the findings of the open card sort to create the predefined categories.
   
   
3. We closed out the research program with a tree test to validate how well the proposed navigation structure performed.

Over a period of 6 weeks we successfully conducted card sort for information architecture redesign and delivered our client an updated IA they could implement that they knew aligned with the mental model of their customers.

READ MORE: 5 Commonly Used Metrics in User Research, Trends and Innovations that Have an Impact in Your Industry, Choosing the Right Survey Tool for Quantitative UX Research,  Our Researchers Can Join Your Team, IA Issues? Online Card Sorting is Not Enough