Unmoderated usability testing

With unmoderated usability testing, there's no researcher to observe the test in a live setting. The only person present in the session is the participant, who works through a pre-defined list of tasks and questions that you've assigned them. 

A great benefit of unmoderated testing is that the user can perform their tasks at any time of their choosing, freeing up your time to focus on other projects as your research project effectively runs in the background. 

Because you don't have the opportunity to interact with your user, you'll need to be crystal clear in your instructions so that the script, scenarios, and tasks feel logical without any further clarification.

To get your results from an unmoderated test, you'll lean heavily on your usability testing tools, which we discussed in the last chapter. 

If you're running a qualitative study, you'll use recording software to view the session later and analyze the user's thoughts and behaviors. If you're taking the quantitative approach, your tool should automatically collect and analyze the data to generate numerical-based insights. 

When should I use unmoderated usability testing?

If you want quick insights from a wide range of participants, unmoderated usability testing is the method for you. Generally, we advise this form of testing in cases where you need specific answers, want feedback on your solution ASAP, or want to see a participant interacting with your product in their natural environment. 

Unmoderated is ideal for: 

  • Meeting agile demands: Agile is the name of the game today. Engineers and product managers are often waiting on your UX insights, so you need to deliver them stat. In this fast-paced environment, unmoderated studies facilitate the need for speed. 
  • Scaling your research: Statistical significance is gained through larger sample sizes. Because unmoderated testing leaves users to their own devices (literally), you can better scale your project, potentially including hundreds of people!
  • Using quantitative data to cement qualitative insights: Qualitative and quantitative research doesn't have to be mutually exclusive—they can work great together. For example, you could run a qualitative study and discover that a handful of users aren't satisfied with a design feature. You can then use a large-scale quantitative test to prove or disprove this insight. 
  • Democratizing UX research: The demand for UX research is higher than ever. You only have 24 hours in a day, so it's impossible for you and your team to run every project that's requested of you. With unmoderated research, people from other departments and other roles can engage in research. With the right platform and some training, you can empower your colleagues to run their own unmoderated tests at speed and scale. 
  • Finding your tribe: Unmoderated research unlocks the potential for you to gain feedback from participants anywhere in the world. Whether you need a geographically dispersed sample size or need to reach a specific group of people with your product, unmoderated research has your back. 

Pros

  • Without the need for a moderator, these tests are inexpensive to run at scale 
  • The statistical significance of your sample size improves confidence in your insights 
  • Access participants from all over the world—the sky’s the limit! 
  • Empower non-UX folk to conduct their own research projects 

Cons

  • You may need to conduct various rounds of testing to uncover the insights you seek 
  • Statistical insights give you an idea of what users are doing—but you won't understand why without qualitative research 
  • You'll need to be careful about your instructions! Confusing directions or a complex product can undermine your research project, leading to poor results. 

So, there you have it…

All there is to know about moderated and unmoderated testing. As you can see, each strategy has its use cases—so you're best off taking an approach where you use a blend of both. 

Next, we'll walk you through some real-world examples so you can start to contextualize usability testing and get a deeper idea of how to conduct your own project.