It’s not about which methodology is best—it’s about which is most appropriate for your testing
It’s one of the most crucial questions that will shape your usability testing project: whether to conduct moderated or unmoderated usability testing.
At first glance, these two terms can seem pretty intimidating, but they're pretty straightforward.
At a high level, the difference between 'moderated' and 'unmoderated' is whether a researcher (the moderator) oversees the test or the test participant is left to carry out the task without any real-time supervision.
Ok, that's the TL; DR version. Now let's look at each in more detail—along with the benefits and challenges.
Is one method of testing better than the other?
We know what you're thinking. Customers often ask us if moderated usability testing is better than unmoderated testing and vice versa.
In truth, there's no clear-cut answer to this. Both forms of testing are valuable in different contexts.
In a moderated usability test, a researcher will be on hand to facilitate the test in real-time. In a remote setting, the researcher will observe the participant remotely via their testing software tool.
During the test, the moderator will ask the participant questions, guiding them through the session to understand their behavior and discover potential usability issues.
Typically, you'll create a test script ahead of the session as a form of structure—but the perk of being in a live setting is that you can ask questions on the fly. For example, if a user interacts with your solution in an unexpected way, you can find out why there and then.
Another cool thing about moderated testing is that you can invite stakeholders and teammates to observe the session anonymously. From our experience, when stakeholders watch a test first-hand, they start to grasp the benefits of user research and are more likely to champion further projects.
Moderated studies have beneficial use cases throughout the design and product lifecycle. Essentially, moderated is the way to go whenever you want to have a conversation with a user and gain in-depth feedback.
In particular, we advocate moderated testing for:
With unmoderated usability testing, there's no researcher to observe the test in a live setting. The only person present in the session is the participant, who works through a pre-defined list of tasks and questions that you've assigned them.
A great benefit of unmoderated testing is that the user can perform their tasks at any time of their choosing, freeing up your time to focus on other projects as your research project effectively runs in the background.
Because you don't have the opportunity to interact with your user, you'll need to be crystal clear in your instructions so that the script, scenarios, and tasks feel logical without any further clarification.
To get your results from an unmoderated test, you'll lean heavily on your usability testing tools, which we discussed in the last chapter.
If you're running a qualitative study, you'll use recording software to view the session later and analyze the user's thoughts and behaviors. If you're taking the quantitative approach, your tool should automatically collect and analyze the data to generate numerical-based insights.
If you want quick insights from a wide range of participants, unmoderated usability testing is the method for you. Generally, we advise this form of testing in cases where you need specific answers, want feedback on your solution ASAP, or want to see a participant interacting with your product in their natural environment.
Unmoderated is ideal for:
All there is to know about moderated and unmoderated testing. As you can see, each strategy has its use cases—so you're best off taking an approach where you use a blend of both.
Next, we'll walk you through some real-world examples so you can start to contextualize usability testing and get a deeper idea of how to conduct your own project.