What is remote moderated research and what do I need to prepare?
Welcome to our handy guide to remote moderated research and how this methodology delivers impactful UX insight over any distance from any location.
Despite the vast array of UX research techniques and methods within a researcher’s toolkit, it sometimes feels like in-person or lab-based moderated research is the only tool available (or is the only tool your stakeholders care about).
However as the world becomes more concerned about non-essential travel and at the same time has never been better connected through digital technology, it’s time for a serious consideration of the benefits of remote moderated research and testing. A UX methodology that means you can still observe and interview users, without anybody having to leave the house.
What is remote moderated research?
In remote moderated testing, your test participants are observed in real-time by a moderator via screen-share technology.
The moderator watches the user interact with a product, listening to their spoken-out-loud thoughts, and can probe the user further by asking in-the-moment questions. Something you can’t do if you’re watching a recording of the test at a later date (which would be the case with remote unmoderated testing).
Remote moderated testing also means that you and your participant(s) can do this from the comfort of your own homes or locations. It’s the same premise as in-person or in-lab testing (the moderator is there to ask participants questions, respond to questions and feedback, and guide them through the tasks) but you are live online with participants from anywhere in the world, connected to them via online software.
This is one of the primary reasons why conducting remote moderated studies is so beneficial – you get all the quality insights without anybody having to travel.
Remote vs. in-person
Within the world of moderated research there’s traditionally been a strong emphasis on in-person testing, and there are many good reasons for this…
- You can read participants’ body language and facial expressions easily – perhaps frustrations aren’t being communicated verbally, but you can see it on their faces.
- You control the hardware – whatever device you are testing on, it will be your device and so you control it. This is a huge advantage, because technical challenges can become an issue when you’re conducting remote moderated sessions.
- You control the location and the environment – great for you, but maybe a participant would be more relaxed if they were in their own natural setting?
The first two factors don’t need to be an issue when it comes to remote research, and it’s that final point that suggests why in-person testing *may* not provide the most valuable insight… How often do you sit in a stuffy conference room, using an app, while being monitored by a stranger asking you why you’re doing the things you’re doing?
What technology do I need to run remote moderated research?
If you’ve been running nothing but in-lab research, you may be overwhelmed by the online options you that are available to you at varying price-points and capabilities.
Often you may find that a cheaper dedicated UX testing tool will suffice, but you’re having to use a variety of other unintegrated tools to make it work for your needs. For instance, you may have to run a Zoom call or Google Hangout while running the usability test to collect your user’s spoken thoughts and see their behaviour in the moment. You might then have to rely on your your own notes, or record the call and run it through a transcription service in order to have an accurate record of the insights.
Perhaps there’s a solution that does offer all of the above (online usability tests, screen-share technology, automated transcription and reporting) but is just out of your budget-range.
If these issues apply to you and your current situation, may we humbly recommend an alternative… Validately, our qualitative UX insights solution that replicates the lab experience, but with all of the efficiency of an online experience – including automated scheduling, stakeholder collaboration, and geographic reach.
And you can sign-up to a free trial below…
Get your remote lab running and delivering insights within minutes!
Validately is a quick and simple way to transition from lab to remote user research.
You can easily talk and listen to users, collaborate with stakeholders, automate scheduling and reporting – all completely remote, so you can iterate quickly from anywhere in the world and continue making great digital experiences.
Take a free trial today and you’ll receive 5 x 30-minute moderated tests, sourced from our global panel of users. All on us!
The benefits of remote moderated research
As Kuldeep Kelkar and Jamie Miller state in our Remote Moderated 101 ebook, the main reason for running remote moderated research is so that you can be in a live setting with a participant despite their distance from you.
This allows you to have a conversation with your users as you’re observing what they are doing and dig deep into usability issues in order to better understand their behavior, while also saving travel costs.
There are many other benefits too:
- You can modify your script on the fly to probe further if there are confusing areas and ask ad-hoc follow-up questions. It’s important not to ask leading questions to adversely introduce your own bias, but there are advantages of adapting to the needs of the moderator, stakeholders/observers, and the participant on the fly. This will save you the hassle of running another test just to ask follow up questions.
- Geographic distribution. Your target market will probably not be in your exact same location, and one of the most common challenges we hear from researchers is that they find it harder and harder to find the right participants within their geographic area. That’s where remote moderated comes in handy.
- You don’t have to pay travel expenses or provide refreshments. Very important if budgets are tight or non-existent.
- Cloudify your lab. You don’t have the hassle, expense and inconvenience of running an actual usability lab in your office.
- Flexibility. You can be more flexible with participants’ availability, especially it they’re in different timezones.
- Own devices. There’s merit in observing a participant using their own technology rather than the hardware and the technology that is given to them.
- Non verbal communication. Because you’re using a webcam, you’ll be able to read their body language and facial expressions, which often reveal more than words, or at least help clear up any ambiguous phrasing.
Eight things to remember when preparing remote moderated research
Here’s a cheat-sheet to what our in-house experts recommend when it comes to running your own remote moderated research:
1) Add technology questions to your screeners
The first thing you’ll want to do is add technology questions to your screener questions. This could be anything under the sun depending on what you need to run your study successfully. For example, are there certain browser requirements that you need? Do they need to join with Chrome or Firefox?
There are prototypes that only run in one or the other, so if that’s the case make sure to put that upfront and make sure your participants are willing to download one or the other. Or you can just screen them out entirely if that’s too much of a hassle.
Here are some more tips for optimising your remote moderated test script.
2) Be sure to over-recruit
Just as you would for in-lab studies, make sure you over-recruit participants, so that you have the confidence you’ll have enough people in the end.
We typically over-recruit by at least one or two participants, and what you can do is schedule a different day just for your make-ups. Consider telling a couple of participants, “You’re our backup and will get a phone call if we need you; what time slot do you want on your make-up days?”
That or just over-recruit. If you’re shooting toward ten, recruit twelve, and if you get twelve, yay! It depends on your budget and your timeline.
3) Remember to put reminders and tech instructions in your email
Your goal is to make sure your participants understand the technical side of what you need from them in advance. Ask them to install whatever software you’re using well ahead of time, and be very clear and concise in your instructions.
Reminders are key as well, and thankfully there are some tools out there, which you can use to make this easy. Calendly and YouCanBook.me are great, and they have built in reminders that are all automated so you don’t have to even think about that.
4) Make sure you have a dial-in
This is one of the key things that has saved our bacon a couple of times. Imagine that you’re sitting there in your virtual meeting room, waiting for the participant to join and they’re not joining. You’re trying to figure out what’s going on only to learn they’re having issues joining the audio through their computer. So having a dial-in number as a backup is a good idea.
5) Be considerate of time zones
I know this is pretty obvious, but there are a lot of mix-ups when it comes to time zones, especially if you’re in a different country than your participant. So in your reminder and booking confirmation emails, always put the time slot in the participant’s time zone.
We don’t recommend putting multiple time zones in there, just put the participant’s time zone to avoid confusion.
6) Ensure there is enough time between the sessions to reset
You might want to talk to your stakeholders in between sessions or you might have a talker. That way even if you go over your time you’ll have buffer. We recommend 30 minutes to help you reset and get ready for the next session. Depending on how intense the sessions are, you may actually want to make that buffer closer to an hour.
7) Prepare your consent forms
This is something we know a lot of people do – send NDAs and consent forms. If you’re going to do this, we recommend doing so before the session starts, because it takes one more thing out of the equation that you have to worry about.
At the beginning of the session you can double-check to make sure that they did sign those forms and they’re good with the recordings. Sometimes you may get questions about where the video will end up – so be prepared to reassure participants that it will only be used internally.
8) Make rescheduling and cancelling an easy process
Finally, have a way to easily reschedule or cancel. Those tools we mentioned before, like YouCanBook.me, have that all built in. It is pretty slick to be honest. You can go into the original email that was sent to the participant and they can click a button and reschedule or cancel from there. If you’re doing it manually make sure it is going to a human, i.e. yourself or your recruiter, and not to an email@example.com email that you may forget about.
Final very important point to stress…
We have watched and conducted hundreds of these sessions and even then we cannot always anticipate all the problems that might arise. Anything that can go wrong usually does, so you just have to prepare, prepare, prepare beforehand for as many situations as possible to try and mitigate them.
Now you know the benefits of remote user research, learn whether moderated or unmoderated is the right choice for you!
In our brand new mini ebook, we’ve compiled a variety of real-life examples from leading UX practitioners across the globe, all of whom offer their own use-cases on when best to use moderated and unmoderated research.
The ebook also includes
- A concise introduction to both moderated and unmoderated testing
- Key issues to consider when undertaking undertaking usability testing
- A very handy cheat-sheet to the pros and cons of both methods
Download below, it’s completely free and you don’t even have to fill-in a boring form to get it…
Main image by Vidar Nordli-Mathisen
Christopher is the Content Marketing Manager, which basically means the skipper of the good ship ‘UserZoom blog’. So far his requests for changing its name to the ‘USS-erzoom Blog’ have been rightfully denied. In his spare time, Christopher is a filmmaker and the editor of wayward pop culture site Methods Unsound. He used to be the deputy editor of Econsultancy, editor of Search Engine Watch, staff writer for ClickZ and features editor of CMO.com.
Kuldeep Kelkar has more than 15 years of UX research, design and engineering experience and joins UserZoom from TATA Consultancy Services (TCS), where he was Head of User Experience. Prior to that, he led UX Design and User Research at PayPal for nearly a decade. He also has led UX efforts at KLA-Tencor and Icon Medialab.
Kuldeep is a renowned author and speaker, and he brings a BE in Industrial Engineering from Nagpur University and an Masters Degree in Human Computer Interaction from the Industrial Engineering department at Clemson University.
Jamie has nearly a decade of experience conducting qualitative and quantitative research for Fortune 500 companies. In her spare time, she volunteers for a non-profit pet rescue and cheers on the Sharks.