What is remote moderated research and what do I need to prepare?
Welcome to our handy guide to remote moderated research and how this methodology delivers impactful UX insight over any distance from any location.
Despite the vast array of UX research techniques and methods within a researcher’s toolkit, it sometimes feels like in-person or lab-based moderated research is the only tool available, or is at least the only tool your stakeholders care about. However as authors Kuldeep Kelkar, SVP, Global UX Research Services, and Jamie Miller, Senior UX Researcher, point out in their Remote Moderated 101 guide:
“That’s like using a hammer for everything from hanging posters to cutting down a tree. In-person moderated is a powerful tool that can yield great insights but it is far from a panacea for all your user research needs.”
So let’s talk about a method that may not be in your existing user research tool-kit, but could be a lot more beneficial and easier to implement than you think…
What is remote moderated research?
‘Moderated research’ is when you listen to your participants spoken-out-loud thoughts and watch their behaviour as they interact with your product, but while you observe you can dig a little deeper by asking them ad hoc questions in the heat of the moment. Something you can’t do if you’re watching a recording of the test at a later date.
If you add ‘remote’ into the mixture, this basically means that you and your participant(s) can do this from the comfort of your own homes or locations. It’s the same premise as in-person or in-lab testing – the moderator is there to ask participants questions, respond to questions and feedback, and guide them through the tasks – but you are live online with participants from anywhere in the world, connected to them with screen share technology.
This is one of the primary reasons why conducting remote moderated studies is so beneficial – you get all the quality insights without anybody having to travel.
Remote vs. in-person
As we mentioned near the beginning of this article, within the world of moderated research there’s historically been a strong emphasis on in-person testing, and there are many good reasons for this…
- You can read participants’ body language and facial expressions easily – perhaps frustrations aren’t being communicated verbally, but you can see it on their faces.
- You control the hardware – whatever device you are testing on, it will be your device and so you control it. This is a huge advantage, because technical challenges can become an issue when you’re conducting remote moderated sessions.
- You control the location and the environment – great for you, but maybe a participant would be more relaxed if they were in their own natural setting?
It’s that final point that suggests why in-person testing *may* not provide the most valuable insight. How often do you sit in a stuffy conference room, using an app, while being monitored by a stranger asking you why you’re doing the things you’re doing?
The benefits of remote moderated research
As Kuldeep and Jamie state in their e-book, the main reason for running remote moderated research is so that you can be in a live setting with a participant despite their distance from you. This allows you to have a conversation with your users as you’re observing what they are doing and dig deep into usability issues in order to better understand their behavior, while also saving travel costs.
- You can modify your script on the fly to probe further if there are confusing areas and ask ad-hoc follow-up questions. It’s important not to ask leading questions to adversely introduce your own bias, but there are advantages of adapting to the needs of the moderator, stakeholders/observers, and the participant on the fly. This will save you the hassle of running another test just to ask follow up questions.
- Geographic distribution is important – your target market will probably not be in your exact same location, and one of the most common challenges we hear from researchers is that they find it harder and harder to find the right participants within their geographic area. That’s where remote moderated comes in handy.
- You don’t have to pay travel expenses, or provide refreshments – very important if budgets are tight or non-existent.
- You don’t have the hassle, expense and inconvenience of running a lab in your office.
- You can be more flexible with participants’ time needs, especially it they’re in different timezones.
- There is also merit in observing a participant using their own technology rather than the hardware and the technology that is given to them.
- If you’re using webcams, you’ll still be able to read their body language and pick up on any non-verbal communication.
For a more in-depth, practical guide to successfully planning and conducting Remote Moderated Research check out our free-to-download, comprehensive ebook:
Eight things to remember when preparing remote moderated research
It’s a common mantra in user research, but the quality and relevance of the people you choose to participate in your studies really does matter when it comes to revealing valuable UX insights.
But once you’ve taken the key components of recruitment into account (such as sample size and finding the ‘right’ participants) there are still things that you should definitely prepare for when recruiting and running your research, particularly when it comes to remote moderated.
1) Add technology questions to your screeners
The first thing you’ll want to do is add technology questions to your screener questions. This could be anything under the sun depending on what you need to run your study successfully. For example, are there certain browser requirements that you need? Do they need to join with Chrome or Firefox?
There are prototypes that only run in one or the other, so if that’s the case make sure to put that upfront and make sure your participants are willing to download one or the other. Or you can just screen them out entirely if that’s too much of a hassle.
2) Be sure to over-recruit
Just as you would for in-lab studies, make sure you over-recruit participants, so that you have the confidence you’ll have enough people in the end.
We typically over-recruit by at least one or two participants, and what you can do is schedule a different day just for your make-ups. Consider telling a couple of participants, “You’re our backup and will get a phone call if we need you; what time slot do you want on your make-up days?”
That or just over-recruit. If you’re shooting toward ten, recruit twelve, and if you get twelve, yay! It depends on your budget and your timeline.
3) Remember to put reminders and tech instructions in your email
Your goal is to make sure your participants understand the technical side of what you need from them in advance. Ask them to install Zoom or WebEx, for example, or make sure they’re running X on Chrome.
Reminders are key as well, and thankfully there are some tools out there, which you can use to make this easy. Calendly and YouCanBook.me are great, and they have built in reminders that are all automated so you don’t have to even think about that.
4) Make sure you have a dial-in
This is one of the key things that has saved our bacon a couple of times. Imagine that you’re sitting there in your WebEx or your Zoom or whatever meeting room, waiting for the participant to join and they’re not joining. You’re trying to figure out what’s going on only to learn they’re having issues joining the audio through their computer. So having a dial-in number as a backup is a good idea.
5) Be considerate of time zones
I know this is pretty obvious, but there are a lot of mix-ups when it comes to time zones, especially if you’re in a different country than your participant. So in your reminder and booking confirmation emails, always put the time slot in the participant’s time zone.
We don’t recommend putting multiple time zones in there, just put the participant’s time zone to avoid confusion.
6) Ensure there is enough time between the sessions to reset
You might want to talk to your stakeholders in between sessions or you might have a talker. That way even if you go over your time you’ll have buffer. We recommend 30 minutes to help you reset and get ready for the next session. Depending on how intense the sessions are, you may actually want to make that buffer closer to an hour.
7) Prepare your consent forms
This is something we know a lot of people do – send NDAs and consent forms. If you’re going to do this, we recommend doing so before the session starts, because it takes one more thing out of the equation that you have to worry about.
At the beginning of the session you can double-check to make sure that they did sign those forms and they’re good with the recordings. As a joke, I say “Don’t worry we’re not going to put you on YouTube” because I sometimes get questions about where the video will end up – so be prepared to reassure participants that it will only be used internally.
8) Make rescheduling and cancelling an easy process
Finally, have a way to easily reschedule or cancel. Those tools we mentioned before, like YouCanBook.me, have that all built in. It is pretty slick to be honest. You can go into the original email that was sent to the participant and they can click a button and reschedule or cancel from there. If you’re doing it manually make sure it is going to a human, i.e. yourself or your recruiter, and not to an firstname.lastname@example.org email that you may forget about.
Just to stress the point on preparation: We have watched and conducted hundreds of these sessions and even then we cannot always anticipate all the problems that might arise. Anything that can go wrong usually does, so you just have to prepare, prepare, prepare beforehand for as many situations as possible to try and mitigate them.
If you spend too much time arguing with stakeholders about whether to run unmoderated or moderated testing, it’s time to put the argument to rest with the help of a furry red monster and some empirical data.
Join us on January 23 for a webinar hosted by Asha Fereydouni, Sr. UX Researcher and Research Partner at UserZoom, where he proclaims “ELMO (enough let’s move on)!” to the great unmoderated/moderated debate using actual evidence and insight based on his own research.
Discover the results by signing up below!
Christopher is the Content Marketing Manager, which basically means the skipper of the good ship ‘UserZoom blog’. So far his requests for changing its name to the ‘USS-erzoom Blog’ have been rightfully denied. In his spare time, Christopher is a filmmaker and the editor of wayward pop culture site Methods Unsound. He used to be the deputy editor of Econsultancy, editor of Search Engine Watch, staff writer for ClickZ and features editor of CMO.com.