What is Card Sorting?
All too often information on websites is organized according to stakeholder perceptions. As User Experience professionals, our role is to bring focus back onto the perceptions and desires of the users to provide an optimal balance between the two. Card sorting is an established technique often used to help structure, organize, and generally improve the “findability of content or functionality” on a website, (Rubin and Chisnell, 2008, p. 18).
Card sorts help improve user access to information by making site interaction more efficient. Users come to a website with expectations of how they will interact with a site and the language and terminology they will find there. Card sorts help improve the way information is structured on the site so that it matches users’ mental models. A poor match can result in an unsatisfactory user experience and visitor drop off. A good match can reduce bounce rate and increase visitor satisfaction. Card sorting studies provide insights during a new design or redesign of a site.
2. What is Online Unmoderated Card Sorting?
Traditional individual card sorting involves a researcher meeting a participant in a structured environment, often a usability lab, presenting them with a deck of 3X5 cards and watching them place those cards into groups according to the design of the study. While traditional card sorting is a tried and true tool in the UX toolkit, it can be limited by budgetary and time constraints.
Today User Experience professionals operate in a faster paced research environment. We work with more complex information architecture. We often feel the need to conduct increasing numbers of studies. As such, robust technology has been created to run high quality online card sorts and to collect and analyze increasing amounts of data.
Benefits of Online Card Sorting
- More sorts in less time: Conduct more sorts in less time, because your online studies are up and running quickly. These are less resource intensive and provide easier, automated data collection and analysis.
- More participants: You can test higher numbers of participants for increased confidence level in data. While resource restraints often limit lab-based card sorts to roughly fifteen participants, UserZoom card sorts easily and typically include 50 to 60 participants.
- Reduce Cost: There is no need to rent lab space or compensate participants for travel expenses. Researchers aren’t required to moderate sessions.
- Faster data analysis: Analyze your data is faster. Heavy statistical analysis and data representation is conducted instantly. Save yourself the time of manually analyzing data.
3. Card Sorting Techniques: Open Sort and Closed Sort
Open and closed card sort techniques have different purposes. Open cards sorts are generative. They help to develop information organization, site structures, terminology etc. Closed card sorts are evaluative. They are used to help verify or confirm information organization and site structures in brand new design and redesign efforts.
1. Open Card Sorting
How does an open card sort work?
- In a remote open card sorting activity, participants are presented with a list of items.
- Participants are asked to sort the list into groups according to how they feel the items should be organized and by their relation.
- Items that participants feel do not belong in any group are identified as well.
- Participants are then asked to provide appropriate category titles for the groups they have made, as well as, describe these groupings.
What can you learn from an open card sort?
- How your target customers expect to see information organized and navigation structured on your site.
- Ideas for new or more appropriate categories and naming conventions.
- How users see the relationship between items.
2. Closed Card Sorting
Closed card sorts are conducted for many reasons including: to help confirm the appropriateness of categories derived from an open card sort, to evaluate the vocabulary, organization, and structure of an existing site, and to determine the best way to add more content into an existing category structure.
How does a closed card sort work?
- In a remote closed card sorting activity, participants are presented with a list of items and a list of categories (menu items).
- Participants are asked to sort the list into one of the provided categories according to how they feel the items should be organized.
- An additional category ‘Does not belong in any group’ may be provided so that items that participants feel do not fit within any group are identified as well.
- Participants are also asked to identify which items they didn’t understand and why.
What can you learn from a closed card sort?
- How your target customers expect to see information organized and navigation structured on your site
- How users see the relationship between items
- What items, if any, participants feel are outliers and do not fit into established categories.
[Fig. 1 Example of UserZoom Open Card Sort Task]
Use open and closed card sorting in tandem
Using open and closed sorts in tandem is beneficial. We recently helped a client gain insight into how visitors perceived the information architecture on its site as part of a redesign project. They started with an open sort to determine user created categories. Later, they followed with a closed sort to evaluate redesign changes made to the site (some of which were suggested by the initial open card sort). The closed card sort revealed a high overlap between users expectations and the redesign organization. In this way, the closed sort supported the open sort results.
Additionally, following an open card sort – multiple closed card sorts (with different sets of users) can be conducted in parallel to test and compare multiple ideas of reorganizing etc information on a site (Tullis 2007, cited in Tullis and Albert, 2008, p. 225).
4. What metrics can be collected with card sorts?
Two important metrics that can be collected from closed (evaluative) card sorts include:
1. The percentage of items that participants place into the appropriate category (Tullis and Albert 2008, p. 51). This metric is indicative of the degree of match between users mental model and a site’s structure and organization. UserZoom captures this metric as ‘Frequency by category’.
[Fig. 2 Screenshot of UserZoom Frequency by Category]
2. The highest percentage for each item. In other words, “this is an indicator of how well the ‘winning’ group pulled the appropriate cards to it” (Tullis and Albert, 2008, p. 51). Was the item placed almost solely in one category (ideal), split evenly among few or more categories (not ideal) or not placed into any category at all? This metric is captured in UserZoom as the ‘Frequency by item’ data display.
[Fig. 3 Screenshot of UserZoom Frequency by Item]
5. Making sense of card sort data
Card sort analytical tools characterize “‘perceived differences’ among pairs of all cards in [a] study” (Tullis & Albert, 2008, p. 218). UserZoom not only captures the metrics already mentioned above (item frequency by category, frequency by item), it also offers access to visual diagram data in the form of a dendrogram.
Dendrogram
Dendrograms, or tree diagrams, can be generated from card sorting data (using hierarchical cluster analysis) to visually show the groups of items based on participant’s perceptions of their relationships.
[Fig. 4 UserZoom Dendrogram]
A dendrogram is a branching diagram illustrating the strength of relationships between items and between groups of items. When interpreting dendrograms to identify potential new groupings of information, look for clusters of items that are both distinct and compact. Compactness refers to the observation that the shorter the distance between two items or groups of items, the more similarity they share. Distinctness refers to the observation that the longer the distance between items or groups of items, the more distinct they are perceived to be from one another.
6. Online Card Sorting with UserZoom
There are various CS tools available in the market. UserZoom’s online research platform offers, among other products, an online card sorting solution for User Experience professionals. With UserZoom’s Card Sorting tool you can:
- easily test large numbers of users (we recommend 50)
- optionally recruit actual visitors to your site (or certain pages of your site) in real time through intercepts
- create a study with up to 100 items to sort out & up to 12 categories to group
- present participants with an intuitive and easy to use drag and drop interface
- monitor results as they come in
- conduct additional data analysis with SPSS and other compatible tools
Another important feature of the UserZoom card sorting solution is the ability to collect participant categorization criteria. You can ask participants questions and additionally ask follow up questions depending on their responses. This allows participants to answer the important question: “Why?”. One perception of online unmoderated card sort studies is that they produce only quantitative data at the expense of qualitative data. In UserZoom, however, open-ended questions can be included so that users can explain their category selections and their rationale for sorting items. They can also identify terms/categories that they did not understand. Additionally, users can evaluate ease or difficulty of each sorting activity.
In open sorts conducted in UserZoom, participants are requested to write descriptions of the categories they have created. Not only is this information highly useful on its own, UserZoom makes it easy to capture and organize this information for viewing at a glance across many participants. You can select items from a checkboxed list. Depending on which items you select, you will see category descriptions written by participants who placed those items into the same category. This feature saves many hours of effort that would be required to do this manually.
[Fig. 5 Screenshot of UserZoom Category descriptions]
7. Conclusion
Online cardsorting is becoming an increasingly common part of a User Experience proffesional’s toolkit.. In today’s fast paced global economy, sites are designed for more diverse audiences, and have increased complexity. Business needs dictate that oftentimes more must be done with less (time and money). As such there are increasing needs for more robust tools to meet these challenges. UserZoom’s card sorting solution offers rich features to meet those needs. You can use UserZoom to conduct sorts on a variety of content (menus, site terminology, category headers, products, etc). With online card sorting tools such as UserZoom, you now can have the best of both worlds — powerful quantitative metrics collection and data analysis, along with insight into participants’ expectations.
