Quant & Qual in a Single Study – The Perfect Coupling
If you’re only measuring one, you’re only getting half the picture.
Quantitative and qualitative data make quite the odd couple. Quantitative data is the prim, by-the-numbers type while qualitative is their emotion driven, speaks-their-mind counterpart. For UX and Market Researchers, both are extremely valuable sets of data to have because both are telling part of the user experience. The key word here is part of the user experience.
What better way to get the full picture of your users’ journeys, while getting as much actionable insights as possible from every participant, than by combining quant and qual in the same study?
If you’re reading this article on a UX blog, I’m willing to bet you already know what each type of data is so please bear with me for a moment just in case.
Quantitative data are the numerical insights that are collected during a study such as task completion rates, time on task, page views, clicks, et cetera. In laymen’s terms this is the What Happened data and while this is certainly helpful, it’s difficult to analyze due to the lack of context. For example, you might know that a user spent 5 minutes on a page and clicked 20 times – were they enthralled at how magnificent the page was or were they searching in vain and getting frustrated?
Qualitative insights are the descriptive Why data from users, such as audio, video and facial recordings as well as open ended comments. This is where the users themselves give you feedback on what they’re feeling – the key component that numbers can’t tell you. Companies tend to gather qual data in smaller quantities however, since this data can be time consuming to analyze when scaled to large sample sizes. Unfortunately this trade off means less statistical significance.
Combining Quant and Qual
Due to the inherent pros and cons of each, it makes sense to collect both types of data and combine them for an even more robust set of data that tells the complete story by giving you the What and the Why together. For example, if we go back to the scenario of a participant who spent several minutes on a webpage we would be able to see their screen recording, watch their face, hear their think-aloud audio response, and read their open-ended comments post task in order to understand whether they were having a positive experience or a negative experience.
The best part is that collecting quant and qual data in the same study with advanced software allows you to filter down through your results to better understand personas or subsets. If N=200 there’s small likelihood you’re going to have the time to sit down and watch all those videos, especially in Agile timeframes. But if you can filter your data to specific users, such as users who abandoned your task or only users who spent more than X amount of time on a task or a combination thereof, you can make informed decisions while remaining statistically significant with your sample size.
User testing software, and the data they collect, are tools in a toolbox that help you optimize the user experience of your site or app. Sometimes you only need one tool, but other times you might need the entire toolbox. Whether it’s just quantitative data, qualitative videos, or a combo of both it pays to collect both types of data so you have different tools at the ready to make an informed decision based on actual users.
Phil got his degree in creative writing, where they told him he most likely wouldn’t be able to use his degree for his career. He obviously won that round. When not working with UX researchers he can be found teaching martial arts and working on his fiction novels.