5 must dos when running unmoderated remote usability testing

Unmoderated remote usability testing can be tricky, but if done right, it can offer a great deal of valuable insight into providing an exceptional user experience.  Many UX Professionals are new to unmoderated remote usability testing (URUT), so we have provided some quick tips to help you obtain the best results possible.

1. Take control of the situation when building a study

Although you test participants remotely and can’t see them interacting, don’t worry! With the right set of scripting (or test design) features and capabilities you can actually take full control of the situation. For example, when building a study, you must ensure a participant accurately completes a certain task through validation.  This can be accomplished by using either of 2 different methods.  The first validation technique is to ask a question related to the task in order to validate if the participant has successfully completed the task.  The second method is to validate by URL when they have successfully completed their task.  This way you’ll ensure that the effectiveness ratios will be correct and the data collected from hundreds of users is valid.

Furthermore, having the ability to employ advanced scripting capabilities, such as logic, branching, piping, system text customization, randomization, and participant completes control, helps to put the research in control by providing flexible and scalable testing features.

2. Recruitment options & sample quality control

No matter how powerful the technology, the quality and validity of your results will depend upon the participants you are able to recruit for your study.  You must pay a lot of attention to how you recruit participants and how you collect data from these participants. Having multiple methods to recruit and invite users is important to obtaining the results you need. Consider intercepting your real visitors with a simple and direct invitation. It is important to have the ability to screen out cheaters (participants who click at random and provide little or no feedback), speeders (participants who spend less than the expected time on the task).  You should also have the ability to manually exclude outlying participants or participants who do not match your given profile.

3. Pre-test before you go live

URUT can be tricky but having the ability to pre-test allows you to be more comfortable before you launch the study. Always perform pre-testing with your internal team, friends, and family, to ensure your test is working properly and you haven’t missed anything.

4. The right method and the right metrics

With URUT you have the ability to collect quantitative metrics such as effectiveness, efficiency, success, and abandonment ratios. You can also obtain behavioral data such as clickstreams and click heatmaps. For example, UserZoom technology is based on ISO-9241-11 standard, which establish the definition of usability objectives that shall be evaluated. How effective (success and error) and efficient (time and clicks) were they when completing the task, and were they successful with the task or was there a high rate of abandonment due to frustration and/or errors.

5. Data analysis & reporting

Once you collect your results, it is important to have the ability to manipulate and analyze your results so you can turn valid data into actionable results.  Having the advanced analytics capabilities such as filtering, clickstream management, manual exclusion, data aggregation, charts, and graphs, is integral to increasing the specificity and validity of your data.  Since you will be collecting results from a large samples size, it is extremely important to have the ability to narrow your results by filtering results.  If you are using the right software solution, you will be able to look more closely and determine why outliers didn’t complete the given task or why they went off track.  Finally, your software should give you the capability of exporting your data to the program of your choice and create compelling reports.  After all, you want to keep this data for later in case you want to do benchmarking.


Leave a Reply

Your email address will not be published. Required fields are marked *