Benefits of Remote User Testing
Remote User Testing conducted without a moderator also known as unmoderated (a.k.a. automated) remote usability research is a solution born out of market demand, such as:
- “I’d like to obtain statistically significant data in order to quantify usability and user experience. Traditional usability testing in a lab has value in some situations, but the sample is small and it only allows for usability problem identification, not quantification.”
- “I’d like users to participate in their natural context, at home, using their own PCs.”
- “I have an international audience and would like to test in several countries, but I don’t have the budget or the time to conduct a traditional moderated study in a lab”.
- “One of the challenges for us now is to be able to do benchmarking. We’d like to compare different competing sites and see who is performing best, offering the best usability and user experience”.
- “Web analytics tools give us lots of data about what happened on our site at an aggregate level, but we still don’t know why it happened or how it relates to different user goals. We want to know users’ real goals and why they do what they do.”
In order to meet all these requirements, it was necessary to develop tools and research methods that provide insights and data in a different way than traditional usability research methods.
Technological advances have enabled researchers to get rich data from an international audience via the Internet.
One set of advanced techniques is called unmoderated remote user research, and it can be described like this:
- A research solution that uses technology to allow researchers to manage online (remote) usability studies.
- The studies can have various forms, including task-based usability testing of a web application or prototype where a test script is predefined before users are invited to participate. An alternative is to let participants follow their own goals, called ‘true intent testing’, or to let users start from a blank page and give them a specific goal to be accomplished, called ‘free search testing’.
- It allows testing over a large user base, geographically dispersed, so it’s ideal for both nation-wide and international testing.
- Users can participate at any time of day and in their natural context (no moderation is needed), using their own PCs and behaving as they normally would.
- A special kind of software asks users to complete tasks and asks specific questions related to the task and the usability of the site.
- The same software collects session data as users interact with the site. The interaction data includes within-page interactions, not provided by web analytics tools.
- With the help of online tools, researchers can analyze the data and create a report. Since users do not need to phisically attend the sessions and there is no moderation needed, cost-efficiency is clearly one of the main advantages of URUTs, but it’s not the only one.
For researchers and Web marketers, the ability to quantify usability is very valuable. And many of today’s researchers and online channel managers agree that having users in the lab environment is not the best solution to obtain quantitative data that generalizes to real-world usage.