How to get actionable insights with UserZoom card sorts

Actionable insights rarely come from a single source of data, and card sorts are no exception.

While technology enables more types of data to be collected from card sorts, the data is still, arguably, singular in nature.

In this article I’ll share three ways the UserZoom platform helps you use multiple sources of data to glean actionable insights.

1) Look at time taken by participant characteristics

In open and closed card sorts, UserZoom records how long each participant took to complete the card sort. Time is an indicator of mental effort. More time taken means more mental effort expended by participants to understand each card and where to place it. 

For open card sorts there is additional mental effort spent defining category names.

To get to actionable insights from time data, it is often important to understand differences between groups of users; e.g., novices versus experts. UserZoom allows you to combine card sort and question data into a single study.

Questions such as, “When was the last time you used __________?” or, “How often do you use __________?” could surface differences in median completion times between novices versus experts.

With time and participant characteristic data, you can identify which group, if any, is expending more cognitive effort to understand the cards and determine the relationship between cards. This is a good starting point for identifying differences in each group’s understanding of an information architecture (IA).

Next, you need to understand which words or phrases (i.e., cards) are causing comprehension problems.

2) Add similarity results

For open card sorts, UserZoom’s Similarity Grid calculates how often participants put two cards into any single category*.

For example, if the grid shows 80% at the intersection of basketball and football, this indicates that 80% of participants consider basketball and football to be similar. In other words, you can reasonably expect people will go to the same place to find football and basketball related content.

With the new color shading and intersection highlighting in UserZoom, you’ll be able to quickly find those cards with high and low similarity. Armed with high/low similarity results you may be able to answer part of the comprehension problem that surfaced in the time results.

For example, your novices’ Similarity Grid may have lots of low similarity values. This suggests that they don’t understand the definitions of or relationships between cards. Perhaps the cards are too technical or use too much industry jargon.

Alternatively, your experts’ grid may be the one with several low similarity values, which indicates a misalignment with their current domain knowledge. In either case, you’re much closer to taking actionable steps in designing your IA.

*It is important to note that the Similarity Grid does not show in which or in how many discrete categories these two cards were placed together. Because an open card sort allows participants to create their own categories (and category names), it is quite likely that all your participants who put football and basketball together, put them into unique categories; e.g., team sports, pro sports, balls). So an open card sort will tell you that football and basketball related content should be in the same area, but you likely won’t have a definitive insight about what that area should be called.

3) Do more… in Excel!

The Similarity Grid gives you a wealth of information, especially when combined with other mixed-method data from UserZoom. The latest release, however, now allows you to engineer the results data … in Excel.

Perhaps you want to organize the results in a different way. Or maybe you want to combine the calculated similarity data with data from other sources such as tech support logs. UserZoom’s ‘Copy sheet’ feature copies the Similarity Grid to the clipboard, which can then be pasted directly into Excel. When you paste the Similarity Grid into Excel, you get an exact copy of the grid’s structure and content.

By the way, it’s not just the Similarity Grid results that can be copied and pasted into Excel. All results that are presented in a sheet (i.e., table) format have the ‘Copy sheet’ feature!