Tree Testing 101: A User Researcher’s Guide
So much more than playing ‘Wood You Rather?’ with a Red Maple.
As part of our UX 101 education series, where we discuss the different types of studies and research methodologies you can use with our own user research platform, we’d like to introduce our readers to the exciting world of tree testing!
What is tree testing?
Tree testing allows you to test the findability of content on a website or app by asking participants to find a specific piece of content from the information architecture (IA). This allows you to test your taxonomy with participants and detect usability issues in how your site or app is structured.
Often times tree testing is conducted after a preliminary card sort has been completed. It is because of this that tree testing is sometimes referred to as ‘reverse card sorting’ – but really that’s a bit rude isn’t it? Tree testing has worked hard to get where it is today, not to be overshadowed by the other information architecture testing methodology thankyouverymuch.
What are typical use cases for tree testing?
Tree testing validates the navigation structure of your site or app by answering the following questions:
- How easily can users find information?
- Are we using the right labels and terminology?
- Is our structure optimized for our end users?
- How confident are we about the structure?
As far as when this method is typically used in the product/design lifecycle, we most often see customers implementing tree testing at the following times:
- Validating the results of a card sorting study early on in the design process
- Benchmarking an existing IA
- After a site redesign to benchmark performance against their previous design/structure
How does tree testing work?
First and foremost, it should be pointed out that tree testing is actually done on a simplified text-only version of your IA and not on the site or app itself. This means writing down categories and subcategories as well as every item within each category.
After this is done you will want to decide upon a task you want participants to complete, and then mark each possible correct area (as there may be multiple) where users would be successful if they selected.
For example, in the following tree test participants were asked to select where they felt they would be able to find new shower heads:
Notice how certain categories opened up into further subcategories. Typically you will want to allow users to go down to the lowest subcategories within your tree, whether that’s 3, 4 or 5 levels deep, to ensure that a realistic test is conducted (assuming that’s how your IA is setup.)
That being said – if you are conducting research on a specific subset of your IA (for example, maybe only the “Home” section in the above example) you don’t necessarily have to include the entire IA in your tree.
Practical advice for running a tree test
We typically see tree tests that are ran with 50 participants in order to get statistically significant results. Because of the size these are most often ran in an unmoderated setting, although they can be ran with a moderator if you feel the need to delve deeper into participant choices.
Select tasks that are representative of what users might do on your real site, and write them in a language that makes sense to them. That said, try and avoid using the same language in your task as on your tree, as participants will just scan the tree for these phrases rather than considering all options critically. I.e. ask participants “Where would you look for a shower head” instead of “Where would you look for bathroom hardware.”
You also want to avoid bias in your results which is why we recommend randomizing your tasks as well as keeping the amount of tasks you ask participants to perform in a tree test to under 10. The reason for these is that you don’t want participants to start memorizing the tree, therefore influencing results.
When should you use tree testing?
Here are essential facts to consider while deciding on whether or not tree testing is the right approach for your research goals.
- Quick to set up, run and analyze the results
- Quick and easy for participants (normally 15-20 minutes)
- Participants complete the test based on descriptions and no other cues, therefore it is great for validating the site structure and labelling
- The tasks are real life scenarios so they make sense to participants
- Can be used to validate the results of an open or closed card sort
- Participants complete the test based on descriptions and no other cues – when a user is on a website they use a range of cues to make a decision
- There isn’t an opportunity to probe the reason participants make a decision…unless you use follow up questions or run a moderated test
What results do you get?
The results of your tree test will largely be quantitative in nature. For example:
- % of respondents who reach the correct option in the tree and those that don’t
- Time to complete the task
- First click results
- Number of attempts
- Success % on first attempt
- Success % on multiple attempts
- % of participants who clicked on each branch of the IA
However, if you also ask open-ended follow-up questions (E.g. Was there any content missing you expected to see? Were they any labels which didn’t make sense to you?) you will also have qualitative text fields.
Tips for analyzing your results
The big takeaways come from looking at success %’s for tasks and the paths participants take for each task. For example, if you notice that many people are taking a longer than necessary path instead of a more direct route it implies that the labeling and categorization may be confusing to your users. The same is true if you notice that many people are selecting the wrong area.
You’ll also want to take a look at how long it takes for participants to make a decision – typically speaking the longer it takes the less clear the IA is.
Finally, look at the number of success %’s on first attempts versus the number who are successful after multiple attempts as this also indicates that content is difficult to find or the labelling is unclear to users.
Want to learn even more about Tree Testing?
After finishing the 30min course, you’ll be able to:
- Understand what a Tree Test is and why you should do it
- Determine when is the appropriate time in the product life cycle to conduct a Tree Test
- Know the type of data collected from a Tree Test
- Learn how to create a Tree Test in UserZoom
- Know how to interpret the data collected from a Tree Test
Toby is a Senior UX Researcher at Userzoom, working in the EMEA PS team across multiple industries. When not working Toby is a keen runner and likes to travel.