The pros and cons of quantitative and qualitative research.
In my first few weeks of working for a UX testing company I was put under a rigorous training programme.
Every facet of UX testing was explained to me in detail, with practical examples and demos delivered along with a reading list of all the key UX texts. I’m not saying any of this as some kind of unsubtle recruitment drive (I found the constant running up steps and punching frozen meat to be both humiliating and confusing) but to reveal where I am on my beginner’s journey into UX.
For you see, when discussing the virtues of UX testing, two phrases came up regularly that I could only nod sagely at, then make a mental note to look up the meaning of later… quantitative and qualitative research!
Both terms are as hard to pronounce as they are to spell – which is why you’ll often hear UX practitioners refer to them as “qual” and “quant.” They’ll tell you it’s to help save their precious time, but they’ll be lying.
So with that in mind, here is my ‘no nonsense’ guide to both terms…
This is any kind of investigation, experiment or study where the results can be presented with numerical values. The data you’ll uncover in quantitative research is all to do with ‘how many, how often and/or how much, etc.’
For example, let’s say you’ve gathered 30 people together in a room and you’ve asked them all, “What is your favorite Radiohead album?” You then write down the results and have a tally that may look like this:
This is quantitive research and its resulting data in its simplest form.
Why you went to the trouble and expense of gathering people 30 people in a room just to ask them about their favorite Radiohead album on company time… well that’s something you’ll have to justify to your line manager.
Of course this is a simplification. For more complex examples of quantitive data just take a look inside the Google Analytics of your website – pageviews, sessions, bounce-rate, frequency of visits over time. This is all quantitative data.
For more complex quantitative research projects, you may have to use an Excel spreadsheet to help reveal the patterns in your data. i.e. How many people who say that The King of Limbs is their favorite Radiohead album also haven’t heard any other Radiohead albums?
This is where qualitative research comes in…
This is any kind of investigation, experiment or study where the results aren’t in numerical form. These will be the observations, comments, thoughts and feelings of the participants. You may also hear this referred to as empirical research.
Qualitative research can take the form of conversations, interviews, open-ended questionnaires or focus groups.
For example, let’s say you’ve gathered another 30 people together in a room and you’ve asked them all, “What is your favourite Radiohead album and why?”
You will then have a similar quantitive data as the first experiment, but now with extra insight!
Also, congratulations, you’ve just had 60 people come into your office, specifically to talk to you about Radiohead.
Remote user testing is a good example of qualitative research. If you run a UX test on your website using a participant from one of our panels, you’ll receive videos of those tests in which you can see the user interact with your site and, crucially, you’ll hear their thoughts and feelings spoken out loud as they navigate.
A good example of user testing that utilises both quantitive and qualitative data is card-sorting.
In card sorting, a participant has a pile of cards with a different category, subject or article from your website written on each card. They are then asked to group the cards together in a way that seems logical to them – thereby creating a possible set of menus for your website.
Where there is a pattern in the results, the quantitative data can support any decisions you make. However when you need to find out the reasons why people haven’t grouped stuff together or chosen a particular category name, you can then read or listen to their comments to find out their reasonings.
Card-sorting can also be done remotely online, as you can see from the UserZoom example below…
Hmm, this is a much better example than the Radiohead one. Maybe I should have started with that instead.
Well, it’s too late now.
Please note: I originally tried to find a photo of Thom Yorke weighing an amount of things in one hand against the other hand for my header image, but my research proved disappointing.