What is remote moderated research and what do I need to prepare?

The benefits of remote moderated research

As Kuldeep Kelkar and Jamie Miller state in our Remote Moderated 101 ebook, the main reason for running remote moderated research is so that you can be in a live setting with a participant despite their distance from you.

This allows you to have a conversation with your users as you’re observing what they are doing and dig deep into usability issues in order to better understand their behavior, while also saving travel costs.

There are many other benefits too:

• You can modify your script on the fly to probe further if there are confusing areas and ask ad-hoc follow-up questions. It’s important not to ask leading questions to adversely introduce your own bias, but there are advantages of adapting to the needs of the moderator, stakeholders/observers, and the participant on the fly. This will save you the hassle of running another test just to ask follow up questions.
• Geographic distribution. Your target market will probably not be in your exact same location, and one of the most common challenges we hear from researchers  is that they find it harder and harder to find the right participants within their geographic area. That’s where remote moderated comes in handy.
• You don’t have to pay travel expenses or provide refreshments. Very important if budgets are tight or non-existent.
• Cloudify your lab. You don’t have the hassle, expense and inconvenience of running an actual usability lab in your office.
• Flexibility. You can be more flexible with participants’ availability, especially it they’re in different timezones.
• Own devices. There’s merit in observing a participant using their own technology rather than the hardware and the technology that is given to them.
• Non verbal communication. Because you’re using a webcam, you’ll be able to read their body language and facial expressions, which often reveal more than words, or at least help clear up any ambiguous phrasing.

Eight things to remember when preparing remote moderated research

Here’s a cheat-sheet to what our in-house experts recommend when it comes to running your own remote moderated research:

1) Add technology questions to your screeners

The first thing you’ll want to do is add technology questions to your screener questions. This could be anything under the sun depending on what you need to run your study successfully. For example, are there certain browser requirements that you need? Do they need to join with Chrome or Firefox? 

There are prototypes that only run in one or the other, so if that’s the case make sure to put that upfront and make sure your participants are willing to download one or the other. Or you can just screen them out entirely if that’s too much of a hassle.

2) Be sure to over-recruit

Just as you would for in-lab studies, make sure you over-recruit participants, so that you have the confidence you’ll have enough people in the end. 

We typically over-recruit by at least one or two participants, and what you can do is schedule a different day just for your make-ups. Consider telling a couple of participants, “You’re our backup and will get a phone call if we need you; what time slot do you want on your make-up days?” 

That or just over-recruit. If you’re shooting toward ten, recruit twelve, and if you get twelve, yay! It depends on your budget and your timeline.

3) Remember to put reminders and tech instructions in your email

Your goal is to make sure your participants understand the technical side of what you need from them in advance. Ask them to install whatever software you’re using well ahead of time, and be very clear and concise in your instructions.

Reminders are key as well, and thankfully there are some tools out there, which you can use to make this easy. Calendly and YouCanBook.me are great, and they have built in reminders that are all automated so you don’t have to even think about that.

4) Make sure you have a dial-in 

This is one of the key things that has saved our bacon a couple of times. Imagine that you’re sitting there in your virtual meeting room, waiting for the participant to join and they’re not joining. You’re trying to figure out what’s going on only to learn they’re having issues joining the audio through their computer. So having a dial-in number as a backup is a good idea.

5) Be considerate of time zones

I know this is pretty obvious, but there are a lot of mix-ups when it comes to time zones, especially if you’re in a different country than your participant. So in your reminder and booking confirmation emails, always put the time slot in the participant’s time zone. 

We don’t recommend putting multiple time zones in there, just put the participant’s time zone to avoid confusion.

6) Ensure there is enough time between the sessions to reset

You might want to talk to your stakeholders in between sessions or you might have a talker. That way even if you go over your time you’ll have buffer. We recommend 30 minutes to help you reset and get ready for the next session. Depending on how intense the sessions are, you may actually want to make that buffer closer to an hour.

7) Prepare your consent forms

This is something we know a lot of people do – send NDAs and consent forms. If you’re going to do this, we recommend doing so before the session starts, because it takes one more thing out of the equation that you have to worry about. 

At the beginning of the session you can double-check to make sure that they did sign those forms and they’re good with the recordings. Sometimes you may get questions about where the video will end up – so be prepared to reassure participants that it will only be used internally.

8) Make rescheduling and cancelling an easy process

Finally, have a way to easily reschedule or cancel. Those tools we mentioned before, like YouCanBook.me, have that all built in. It is pretty slick to be honest. You can go into the original email that was sent to the participant and they can click a button and reschedule or cancel from there. If you’re doing it manually make sure it is going to a human, i.e. yourself or your recruiter, and not to an info@company.com email that you may forget about.

Final very important point to stress…

We have watched and conducted hundreds of these sessions and even then we cannot always anticipate all the problems that might arise. Anything that can go wrong usually does, so you just have to prepare, prepare, prepare beforehand for as many situations as possible to try and mitigate them.