When is UX online research surpassing UX lab testing?
Welcome to the future of online research.
If we look back on the last 10 years, we can see that usability testing and online research in general has hardly changed. Although it may seem as though it has improved on a technical level, we continue to use the same screen capture software, and improvements in eye tracking have been gradual and by no means revolutionary.
Other industries such as online marketing undergo continuous change, introducing something new every 2 or 3 years and redefining the pillars it is based on. Over the last years, something similar happened in traditional market research in which according to the annual study of ESOMAR 2012 online marketing accounts for 22% and by this is ranked well above phone interviews with 13%, focus groups (13%) and face to face interviews (11%).
This development is not surprising as user experience research is a much newer discipline than market research. It is therefore understandable that market research was the first field to adapt running online studies. However, current surveys and technology developments suggest a promising future for user experience research.
In the “salary survey” created by UPA a.k.a. User Experience Professional Association in 2007, remote online research without a moderator was not even listed. Two years later, in 2009, 18% of those who were interviewed two years earlier stated that they had used unmoderated usability research in the past year. In 2011, the total number of UX researchers who responded that they conducted Remote Usability Testing climbed to 23%, turning this online research method into the one with the highest percentage growth.
UX Software solutions researchers use for Remote Usability Testing are main drivers behind the expansion of remote user research. User Experience Tools have significantly reduced the gap between qualitative information collected in traditional in-lab research and quantitative data gathered by remote user testing. Not only that, UX software solutions also support researchers in analysing quantitative data in order to better understand qualitative information and facilitate data interpretation.
The most remarkable change has been the ability to record user videos and audio feedback as well as capture the users’ browsing sessions and combine them with quantitative UX metrics such as task success rates, time and clicks per task, or navigation paths. Obtaining the videos of 200 users and being able to filter them by success rates, reasons for abandoning or users who clicked on a particular page, has significantly increased the quality of information collected and combines the best of qualitative and quantitative research.
Although this only refers to information captured in a remote user test, there have been other improvements in areas such as quality control of user data. Being able to automatically screen out users who reply too fast or only generate a certain number of clicks, these factors have led to researchers quickly adopting this user testing method. Other fundamental improvements include the automatic validation of task success upon completing a task, or the ability to use different methodologies such as card sorting, task-based tests or screenshot click testing in the same study.
To say that Remote Usability Testing will surpass lab testing is something few dare to predict, however the question is when will the “surpassing” happen.