What are the different types of usability testing methods? 

It’s not about which methodology is best—it’s about which is most appropriate for your testing

It’s one of the most crucial questions that will shape your usability testing project: whether to conduct moderated or unmoderated usability testing. 

At first glance, these two terms can seem pretty intimidating, but they're pretty straightforward. 

Unmoderated vs. moderated research: What's the difference? 

At a high level, the difference between 'moderated' and 'unmoderated' is whether a researcher (the moderator) oversees the test or the test participant is left to carry out the task without any real-time supervision. 

Ok, that's the TL; DR version. Now let's look at each in more detail—along with the benefits and challenges.  

Moderated usability testing

In a moderated usability test, a researcher will be on hand to facilitate the test in real-time. In a remote setting, the researcher will observe the participant remotely via their testing software tool. 

During the test, the moderator will ask the participant questions, guiding them through the session to understand their behavior and discover potential usability issues. 

Typically, you'll create a test script ahead of the session as a form of structure—but the perk of being in a live setting is that you can ask questions on the fly. For example, if a user interacts with your solution in an unexpected way, you can find out why there and then. 

Another cool thing about moderated testing is that you can invite stakeholders and teammates to observe the session anonymously. From our experience, when stakeholders watch a test first-hand, they start to grasp the benefits of user research and are more likely to champion further projects. 

When should I use moderated usability testing?

Moderated studies have beneficial use cases throughout the design and product lifecycle. Essentially, moderated is the way to go whenever you want to have a conversation with a user and gain in-depth feedback. 

In particular, we advocate moderated testing for: 

• Validating high-level concepts: Whether it's an early-stage prototype or just a one-page concept, you can use moderated studies to validate your ideas with people and gauge whether they're worth pursuing further. 
• Testing complex products: Some solutions are too difficult to explain through written instructions. If you feel like you want to be 'in the room' with your user, moderated testing is your best bet. 
• Formulating and refining your research hypotheses: A considerable part of research is developing a hypothesis about how you expect users to interact with your product - based on historical data and personas. Moderated research allows you to refine these hypotheses granularly. In the long run, this helps you create research projects that are clearly defined and rock solid.
• Creating UX advocates: Inviting stakeholders to observe moderated sessions is a surefire way to showcase the value of usability testing. Trust us, try it. You'll be amazed at the levels of empathy your stakeholders build.  
• Testing logged-in environments: Most websites and applications require users to create a dedicated log-in account, making it challenging to test logged-in settings. Moderated studies can help you to overcome this difficulty. For example, you can have a moderator log in and then pass control over to the tester. 

Pros 

• Great for conducting interviews with users and gathering rich feedback about the customer journey 
• Enables you to understand your participants' thoughts and feelings about your solution—something that isn't achievable with qualitative research methods 
• Video-led discussions allow you to pick up on non-verbal cues like body language and facial expressions, which can generate further insights 
• You can support participants through complex tasks 

Cons

• Testing can be lengthy and time-consuming 
• You need to be wary of asking leading questions, which could skew your results 
• Sample sizes are inherently smaller in moderated testing, so there's often less confidence in the data
• Time intensive and planning heavy, this form of testing tends to be expensive 
  

Unmoderated usability testing

With unmoderated usability testing, there's no researcher to observe the test in a live setting. The only person present in the session is the participant, who works through a pre-defined list of tasks and questions that you've assigned them. 

A great benefit of unmoderated testing is that the user can perform their tasks at any time of their choosing, freeing up your time to focus on other projects as your research project effectively runs in the background. 

Because you don't have the opportunity to interact with your user, you'll need to be crystal clear in your instructions so that the script, scenarios, and tasks feel logical without any further clarification.

To get your results from an unmoderated test, you'll lean heavily on your usability testing tools, which we discussed in the last chapter. 

If you're running a qualitative study, you'll use recording software to view the session later and analyze the user's thoughts and behaviors. If you're taking the quantitative approach, your tool should automatically collect and analyze the data to generate numerical-based insights. 

When should I use unmoderated usability testing?

If you want quick insights from a wide range of participants, unmoderated usability testing is the method for you. Generally, we advise this form of testing in cases where you need specific answers, want feedback on your solution ASAP, or want to see a participant interacting with your product in their natural environment. 

Unmoderated is ideal for: 

• Meeting agile demands: Agile is the name of the game today. Engineers and product managers are often waiting on your UX insights, so you need to deliver them stat. In this fast-paced environment, unmoderated studies facilitate the need for speed. 
• Scaling your research: Statistical significance is gained through larger sample sizes. Because unmoderated testing leaves users to their own devices (literally), you can better scale your project, potentially including hundreds of people!
• Using quantitative data to cement qualitative insights: Qualitative and quantitative research doesn't have to be mutually exclusive—they can work great together. For example, you could run a qualitative study and discover that a handful of users aren't satisfied with a design feature. You can then use a large-scale quantitative test to prove or disprove this insight. 
• Democratizing UX research: The demand for UX research is higher than ever. You only have 24 hours in a day, so it's impossible for you and your team to run every project that's requested of you. With unmoderated research, people from other departments and other roles can engage in research. With the right platform and some training, you can empower your colleagues to run their own unmoderated tests at speed and scale. 
• Finding your tribe: Unmoderated research unlocks the potential for you to gain feedback from participants anywhere in the world. Whether you need a geographically dispersed sample size or need to reach a specific group of people with your product, unmoderated research has your back. 

Pros

• Without the need for a moderator, these tests are inexpensive to run at scale 
• The statistical significance of your sample size improves confidence in your insights 
• Access participants from all over the world—the sky’s the limit! 
• Empower non-UX folk to conduct their own research projects 

Cons

• You may need to conduct various rounds of testing to uncover the insights you seek 
• Statistical insights give you an idea of what users are doing—but you won't understand why without qualitative research 
• You'll need to be careful about your instructions! Confusing directions or a complex product can undermine your research project, leading to poor results. 
  

So, there you have it…

All there is to know about moderated and unmoderated testing. As you can see, each strategy has its use cases—so you're best off taking an approach where you use a blend of both. 

Next, we'll walk you through some real-world examples so you can start to contextualize usability testing and get a deeper idea of how to conduct your own project.