Quantifying usability through unmoderated online user testing.

Why you would use quantitative testing when you are already conducting qualitative research? As with all usability methods, each has their place and purpose. Our UZ method and quantitative measures are not to replace the value you obtain from Think Aloud research. Rather, it complements the research you are already conducting.

Imagine walking into a meeting with key stakeholders and asking them to redesign an entire piece of their website based on 8-10 customers. Do you have everything you need and believe to honestly say that the data you are presenting truly represents the 10,000+ customers that hit your site monthly? Quantitative research fills that gap and backs up your qualitative data.

Consider a scenario where you conduct an online quantitative study with 200 customers on your site. You discover five key critical issues that are consistent with 83% of your sample. That is pretty powerful statistics to bring into your meeting. Now take those five critical issues and roll them into a lab study. You have your targeted statistical key issues that you can now probe with a Think Aloud protocol in a lab study. Back in the meeting you are bringing in not only statistical data to validate your critical changes; you are combining it with the thoughts of customers while they were walking through those issues. What a powerful message you are conveying to those stakeholders as you have done your due diligence when testing.

For researchers it is important that we bring back recommendations that will truly represent the customer population and make a huge impact on a site. Combining quantitative and qualitative research methods can achieve that goal and bring research to the next level of impacting the way customers interact with your site.

Understanding user behavior

Still today web managers do not know enough about their online users and customers. How many online channel managers can openly say that they know WHY users come to their website and back this up with statistical data? Would you also like to know: When users are on their way to accomplish a given goal or objective that relates to their business prior to reaching their website, do they know how users behave, what search strategies they use, and what sites they visit?

Web analytics tells us about WHAT happened on the site: What pages are seen, where visits come from, the time spent on each page, etc. But this does not give you user data that combines users’ actual specific intent and users’ actual behavior based on that intent.

Let’s imagine that Charles owns a car accessories website, aimed mostly at sports carowners. For example, one of his specialties is cutting edge car cleaning products for delicate paints. Wouldn’t it be interesting to know how sports car owners search for these types of products online? Would Charles be interested in knowing what terms users entered in order to find them? How about the sites visited? Did they choose a particular brand or website or did they use a search engine? And most importantly, WHY do they search the way they search? Are they looking for specific information? If so, what kind of information? Through unmoderated online user testing Charles would be able to perform what we call a ‘true intent study’ and a ‘free’ or ‘open’ web search study.

To understand why users actually visit his site, he’d need to conduct a ‘true intent study’, which simply works by inviting users who visitthe site to fill out a very short survey. In time, he’ll have hundreds or thousands of respondents and, most importantly, statistics on users’ purpose of their visit.

In the ‘free web search study, he’d be able to ask 300 sports-car owners to start from a blank browser page and from there look for car-cleaning products for delicate paints. Something like this:

“You own a red Corvette (or similar) and you’d like to keep it looking great every other weekend by hand-washing it yourself. You know it takes a special kind of cleaning product to help you wash the car without scratching it, so it looks and shines like new.

Based on this scenario, try to find a product that would satisfy your current needs. You will start from a blank page and you may freely go wherever you want.”

Charles will be able to understand his customers’ behavior in a way that will help him better position his site on his site. He’ll learn marketing and branding data that will help him in his SEM strategy. He’ll know if his competitors are being recognized as well. He’ll understand his customers’ mental models and what terms they use so he’ll make sure his site will fit in with those.

In this case, user behavior research is different than usability research. Each will gather distinct data. However, both are equally important when it comes to providing your users the best online experience possible.

In conclusion, unmoderated online user research (a.k.a. remote user testing), includes a variety of research methods and will help user experience and marketing professionals get highly valuable and actionable user data.