How to deliver impactful UX insight over any distance, from any location.
Despite the vast array of UX research methods within a researcher’s toolkit, it sometimes feels like in-person or lab-based moderated research is the only tool available (or is the only tool your stakeholders care about).
However as the world becomes more concerned about non-essential travel and at the same time has never been better connected through digital technology, it’s time for a serious consideration of the benefits of remote moderated research and testing. A UX methodology that means you can still observe and interview users, without anybody having to leave the house.
In remote moderated testing, your test participants are observed in real-time by a moderator via screen-share technology.
The moderator watches the user interact with a product, listening to their spoken-out-loud thoughts, and can probe the user further by asking in-the-moment questions. Something you can’t do if you’re watching a recording of the test at a later date (which would be the case with remote unmoderated testing).
Remote moderated testing also means that you and your participant(s) can do this from the comfort of your own homes or locations. It’s the same premise as in-person or in-lab testing (the moderator is there to ask participants questions, respond to questions and feedback, and guide them through the tasks) but you are live online with participants from anywhere in the world, connected to them via online software.
This is one of the primary reasons why conducting remote moderated studies is so beneficial – you get all the quality insights without anybody having to travel.
Within the world of moderated research there’s traditionally been a strong emphasis on in-person testing, and there are many good reasons for this…
The first two factors don’t need to be an issue when it comes to remote research, and it’s that final point that suggests why in-person testing *may* not provide the most valuable insight… How often do you sit in a stuffy conference room, using an app, while being monitored by a stranger asking you why you’re doing the things you’re doing?
As Kuldeep Kelkar and Jamie Miller state in our Remote Moderated 101 ebook, the main reason for running remote moderated research is so that you can be in a live setting with a participant despite their distance from you.
This allows you to have a conversation with your users as you’re observing what they are doing and dig deep into usability issues in order to better understand their behavior, while also saving travel costs.
There are many other benefits too:
Here’s a cheat-sheet to what our in-house experts recommend when it comes to running your own remote moderated research:
The first thing you’ll want to do is add technology questions to your screener questions. This could be anything under the sun depending on what you need to run your study successfully. For example, are there certain browser requirements that you need? Do they need to join with Chrome or Firefox?
There are prototypes that only run in one or the other, so if that’s the case make sure to put that upfront and make sure your participants are willing to download one or the other. Or you can just screen them out entirely if that’s too much of a hassle.
Just as you would for in-lab studies, make sure you over-recruit participants, so that you have the confidence you’ll have enough people in the end.
We typically over-recruit by at least one or two participants, and what you can do is schedule a different day just for your make-ups. Consider telling a couple of participants, “You’re our backup and will get a phone call if we need you; what time slot do you want on your make-up days?”
That or just over-recruit. If you’re shooting toward ten, recruit twelve, and if you get twelve, yay! It depends on your budget and your timeline.
Your goal is to make sure your participants understand the technical side of what you need from them in advance. Ask them to install whatever software you’re using well ahead of time, and be very clear and concise in your instructions.
Reminders are key as well, and thankfully there are some tools out there, which you can use to make this easy. Calendly and YouCanBook.me are great, and they have built in reminders that are all automated so you don’t have to even think about that.
This is one of the key things that has saved our bacon a couple of times. Imagine that you’re sitting there in your virtual meeting room, waiting for the participant to join and they’re not joining. You’re trying to figure out what’s going on only to learn they’re having issues joining the audio through their computer. So having a dial-in number as a backup is a good idea.
I know this is pretty obvious, but there are a lot of mix-ups when it comes to time zones, especially if you’re in a different country than your participant. So in your reminder and booking confirmation emails, always put the time slot in the participant’s time zone.
We don’t recommend putting multiple time zones in there, just put the participant’s time zone to avoid confusion.
You might want to talk to your stakeholders in between sessions or you might have a talker. That way even if you go over your time you’ll have buffer. We recommend 30 minutes to help you reset and get ready for the next session. Depending on how intense the sessions are, you may actually want to make that buffer closer to an hour.
This is something we know a lot of people do – send NDAs and consent forms. If you’re going to do this, we recommend doing so before the session starts, because it takes one more thing out of the equation that you have to worry about.
At the beginning of the session you can double-check to make sure that they did sign those forms and they’re good with the recordings. Sometimes you may get questions about where the video will end up – so be prepared to reassure participants that it will only be used internally.
Finally, have a way to easily reschedule or cancel. Those tools we mentioned before, like YouCanBook.me, have that all built in. It is pretty slick to be honest. You can go into the original email that was sent to the participant and they can click a button and reschedule or cancel from there. If you’re doing it manually make sure it is going to a human, i.e. yourself or your recruiter, and not to an email@example.com email that you may forget about.
We have watched and conducted hundreds of these sessions and even then we cannot always anticipate all the problems that might arise. Anything that can go wrong usually does, so you just have to prepare, prepare, prepare beforehand for as many situations as possible to try and mitigate them.
Now you know the benefits of remote user research, learn whether moderated or unmoderated is the right choice for you!