Here at UserZoom we’re all about bringing the maximum amount of learning to people at any stage of their UX development. Hence we’re bringing you these regular UX beginner’s guides, aimed to help all the newbies of the UX testing world. This is also why we’ve started using words like hence to make us sound learned.

This week: card sorting!

How does card sorting work? What are the benefits? What do you need to be aware of? How can it help improve the usability of your website? What’s the difference between open, closed and hybrid card sorting? WHAT DOES IT DO, FOR THE LOVE OF GOD TELL ME IN PLAIN ENGLISH PLEASE???

Well, when you’ve calmed down a little bit, I’ll answer your questions and then perhaps we can go outside for some fresh air.

What is card sorting?

Card sorting is a test you can run to improve the navigation of your website, and like all the best methods of UX testing it puts the user at the centre of your observations.

In card sorting, participants are presented with a list of items (for example, all the products featured in an online supermarket) and asked to group them in a way that makes the most logical sense to them.

Depending on the type of card sort, participants can also choose names for the groups they’ve put together, forming the potential categories and subcategories of a website.

All of this will hopefully create an easier, more logical way of navigating your site.

card sorting
Example from a UserZoom online card sorting test

What does card sorting have to do with Information Architecture (IA)?

You’ll probably hear the phrase Information Architecture (IA) a lot when you begin testing how people navigate your site, and this simply refers to the way content is presented and accessed from any given page – whether that’s through menus, breadcrumbs, categories, links… whatever takes you from one page to another.

This is how our senior UX researcher Hazel Ho refers to IA:

“Information architecture is the practice of arranging parts of something to make it understandable”

Information Architecture typically focuses on…

  • Structure: the way information is laid out i.e. people should be able to predict where to find what they’re looking for
  • Organisation – grouping information in a way that makes sense to people
  • Labels – ensuring elements are appropriately titled so people can find information

A good IA can help people on your site understand where they are, what’s around, and what to expect.

One of the many ways you can help visitors find their bearings on any page of your website is by offering a clear, comprehensive and logical means of navigation.

This is where card sorting comes in.

What are the benefits of card sorting?

As mentioned earlier, card sorting is a test you can run to validate the effectiveness of your site’s organisation, structuring and labelling.

It will also help you decide how to label your categories and navigation, arrange the subcategories underneath parent categories, decide what needs to go on a homepage and figure out where users are getting lost or confused.

The ultimate benefit is that you’ll be building and improving your navigation by observing how real users will navigate your site and its information architecture, rather than just guessing yourself.

Just because you assume that people will find the ’Fruit and Vegetables’ subcategory under a ‘Fresh’ category, it doesn’t mean people won’t also go looking under ‘Frozen’.

Card sorting is also simple to arrange, the materials needed are cheap to produce and it doesn’t take too long.

What are the disadvantages of card sorting?

According to Boxes and Arrows, there are a few things to be wary of when running card sorting:

  • Naturally the results can vary wildly from person to person, or group to group. Much like any user testing, there’s no guarantee of consistency. But it can give you a solid foundation to build your IA.
  • Although the test itself can be quick, the analysis can be time consuming – particularly if there is little consistency between participants.
  • All the participants have are the surface details of the content (a title or headline for a webpage) they won’t necessarily know what the content is about or understand any nuance between cards – especially if its a niche subject. Bear this in mind when looking for participants.

How do I run a card sorting test?

The beauty of card sorting is that it’s pretty low tech. In fact you only really need some post-it notes, a wall or a large table and some test participants.

Or you can use online software to run the test, saving you the hassle of finding all of the above resources. Here’s our own promo video for running a card sort through UserZoom:

Whether online or offline, you’ll have to decide what type of card sort you want to run. There are three possible types of card sorting…

What’s the difference between open, closed and hybrid card sorting?

Open card sort

This is the most flexible option. Participants are asked to group cards into categories that makes sense to them, and then they label each category in a way that they feel accurately describes the content.

Closed card sort

Here participants sort cards into category groups that you’ve already labelled and defined. For example, this is handy if you’re launching a new page for ‘Fitbits’ and you’re not sure whether to put it under a parent category ‘Technology’ or ‘Sports and Leisure’.

Hybrid card sort

As you’d expect this is a mixture of open and closed. Participants can sort cards into categories you’ve already defined and then create their own categories if they think your categories are a bit rubbish.

Then you have to decide on your technique…

Techniques for card sorting

Usability.gov provides the following advice on the different means of card sorting…

One on one

These are in-person sessions with an observer (also known as moderated). Users are encouraged to express their thoughts out-loud to give a clear picture as to why they are making their decisions. This can be done with physical cards or online software, and you will need a person facilitating/observing.

Group

Another moderated technique, these tests have their participants sort the cards out as a group. The group can be briefed at the beginning and at the end of the session, but they are largely left alone for the duration.

This method can lead to quick decisions, but may also be prone to the pitfalls of group dynamics – i.e. the loudest opinion holds the most sway.

Remote unmoderated

These are online based sessions that require participants to work independently from their own computer. These sessions can be done using open or closed methods.

The sessions are recorded by screen-capture software, and the videos can be analysed at a later date (this is what’s known as ‘unmoderated’)

How to prepare your cards

You’ll want to create around 50 cards. Fewer cards may not generate enough groupings, more cards will probably lead to fatigue/boredom.

Below is an estimate on the time required to sort a specific number of items.

  • 20 minutes for 30 cards
  • 30 minutes for 50 cards
  • 60 minutes for 100 cards

You then have to decide on the content of your cards. You can do this by brainstorming the types of information you might want to include on your website, or by looking at your existing product inventory, or by looking at what your competitors have on their websites, or even better – ask your users what information they expect to find.

Once you have all your ideas written down on a lovely spreadsheet, you can refine the possibilities until you’re left with only the most relevant cards.

And finally, you’ll have to find some participants…

How to find the right participants

We recommend testing 50 participants per card sort. 50 participants will allow you the flexibility to exclude participants while maintaining reliable results.

Whether you’re running one-to-one or group sessions, it’s a good idea to recruit participants who will give you real-world data.

Things to bear in mind when working with participants

  • If people aren’t receiving an incentive or are not obligated to participate, you’ll need to cast your net wider.
  • If you’re using physical cards for a group sort, make sure you have enough room for people to spread out the cards and not crash into one another.
  • Let participants know how long the sort should take, so it will help them gauge how much time and effort they should put in.
  • For goodness sake, if you’re handwriting the cards, make sure your writing is legible. Better yet, print them out.

Alternatively you can use an online platform (*cough cough*) who can source the participants for you.

More pro tips for card sorting

Here’s a collection of pro tips that you should remember when running card sorting sessions, rounded off with a a few ‘pro-pro tips’ from our very own Lee Duddell.

  • It will help to number the cards individually on the corner, as you’ll need this for later analysis.
  • Remember that you might have to nudge your participants to think out loud.
  • If you’re running concurrent group or independent sessions, remember you’ll need to provide enough sets of cards.
  • Just running quantitive test will only get you so far, you need qualitative data (the spoken out-loud observations of the participants) to fill in any gaps of understanding.
  • Run card sorts in local languages. Qualitative studies in open card sorts that take into account the nuances of local language will be valuable in ensuring your navigation is understood in international versions of your site.
  • It’s worth remembering that top level menus communicate to people what they can do on a site. They form a major part of a new visitors’ first impression, therefore it’s worth running tests on these.
  • Menus are really hard to A/B test. It’s difficult to drive decent volumes of traffic to split test different configurations of IA, therefore the qualitative approach of card sorting is necessary for menus.
  • You will need to run a tree-test to validate your analysis.

How to analyse your card sort results

If you’ve used online card-sorting software, this should generate a report for you. UserZoom can generate a visual diagram data in the form of a dendrogram.

dendrogram

A dendrogram is a branching diagram illustrating the strength of relationships between items and between groups of items. They help to visually show the groups of items based on participant’s perceptions of their relationships.

When interpreting dendrograms to identify potential new groupings, look for clusters of items that are both distinct and compact. For instance, the shorter the distance between two items or groups of items, the more similarity they share. However the longer the distance between items or groups of items, the more distinct they are perceived to be from one another.

If you’ve used physical cards – you’ll need to analyse the results yourself.

Usability.gov gives the following advice, which I’ve distilled here:

  • After every sort remember to photograph the layout of the cards.
  • Write down the names the participant gave to each grouping (if it’s an open sort) and the corresponding numbers of the cards included under that name.
    • Remember to reshuffle the cards for the next sort.
  • Write down qualitative information in the form of user comments.
  • Write down quantitative information based on: which cards appeared together most often, how often cards appeared in specific categories.
  • It may also be worth noting how much time it took to complete the sort.

You can use this data to find commonalities between the card sorts, or if you have a larger number of cards and sessions you could use an Excel spreadsheet to reveal the relationships and patterns between the cards.

The you should hopefully have enough information to begin structuring (or restructuring) your site’s information architecture.

Main image by Fahan Siddicq.