What’s the difference between quantitative and qualitative research?

In my first few weeks of working for a UX testing company I was put under a rigorous training programme.

Every facet of UX testing was explained to me in detail, with practical examples and demos delivered along with a reading list of all the key UX texts. I’m not saying any of this as some kind of unsubtle recruitment drive (I found the constant running up steps and punching frozen meat to be both humiliating and confusing) but to reveal where I am on my beginner’s journey into UX.

For you see, when discussing the virtues of UX testing, two phrases came up regularly that I could only nod sagely at, then make a mental note to look up the meaning of later… quantitative and qualitative research!

Both terms are as hard to pronounce as they are to spell – which is why you’ll often hear UX practitioners refer to them as “qual” and “quant.” They’ll tell you it’s to help save their precious time, but they’ll be lying.

So with that in mind, here is my ‘no nonsense’ guide to both terms…

What is quantitative research?

This is any kind of investigation, experiment or study where the results can be presented with numerical values. The data you’ll uncover in quantitative research is all to do with ‘how many, how often and/or how much, etc.’

For example, let’s say you’ve gathered 30 people together in a room and you’ve asked them all, “What is your favorite Radiohead album?” You then write down the results and have a tally that may look like this:

• OK Computer: 8
• Kid A: 6
• The Bends: 4
• In Rainbows: 4
• A Moon Shaped Pool: 3
• Amnesiac: 2
• Hail to the Thief: 1
• The King of Limbs: 1
• Pablo Honey: 1 (participant expelled from group for being a ‘contrarian and/or joker’)

This is quantitive research and its resulting data in its simplest form.

Why you went to the trouble and expense of gathering people 30 people in a room just to ask them about their favorite Radiohead album on company time… well that’s something you’ll have to justify to your line manager.

Of course this is a simplification. For more complex examples of quantitive data just take a look inside the Google Analytics of your website – pageviews, sessions, bounce-rate, frequency of visits over time. This is all quantitative data.

For more complex quantitative research projects, you may have to use an Excel spreadsheet to help reveal the patterns in your data. i.e. How many people who say that The King of Limbs is their favorite Radiohead album also haven’t heard any other Radiohead albums?

Pros of quantitative research

• Quantitative data is relatively easy to gather, doesn’t necessarily take very long and you’ll find it easier to assemble a larger group of respondents.
• Fast results
• Quantitive research tends to be objective (i.e. it’s harder to ‘lead’ participants with more complex questions)
• It’s easy to make nice visualisations from the raw data
• People love statistics!

Cons of quantitative research

• A large sample size is needed for any kind of statistical significance. It’s no good asking only 3 people their favorite Radiohead album, when there are 6 more albums than people who you’re asking.
• Quantitive data can tell you what is happening and how many times – it can’t tell you WHY it’s happening.
• If you want to use quantitive data to make changes or improve your product or website, you’ll only be making a guess as to what those changes need to be.

This is where qualitative research comes in…

What is qualitative research?

This is any kind of investigation, experiment or study where the results aren’t in numerical form. These will be the observations, comments, thoughts and feelings of the participants. You may also hear this referred to as empirical research.

Qualitative research can take the form of conversations, interviews, open-ended questionnaires or focus groups.

For example, let’s say you’ve gathered another 30 people together in a room and you’ve asked them all, “What is your favourite Radiohead album and why?”

You will then have a similar quantitive data as the first experiment, but now with extra insight!

• You may discover that although eight people loved OK Computer, half of them only like it because it’s the album they heard the most.
• One person might say something about it “perfectly capturing the air of scepticism around the 1997 general election.”
• Someone might appreciate it for the “multilayered arpeggiated guitars and experimental time signatures.”
• Someone else may just shrug their shoulders and make a farty noise with their mouth.
• Either way, you’ve learned something about your participants, the topic of your study and their relationship to one another.

Also, congratulations, you’ve just had 60 people come into your office, specifically to talk to you about Radiohead.

Remote user testing is a good example of qualitative research. If you run a UX test on your website using a participant from one of our panels, you’ll receive videos of those tests in which you can see the user interact with your site and, crucially, you’ll hear their thoughts and feelings spoken out loud as they navigate.

Pros of qualitative research

• You’ll be inside the mind of the person using your product
• You should hopefully see things that quantitive data can’t reveal – for instance if there’s a page on your website losing traffic, you’ll be able to witness first hand what happens when a genuine user visits the page
• Participants may find it easier to reveal their feelings about something, rather than assigning a number or ticking a ‘yes or no’ box.
• This anecdotal evidence can be more persuasive than hard data – especially if it’s emotionally driven and your stakeholders can see and hear these observations for themselves.

Cons of qualitative research

• Because of the time and resources involved, qualitative research tends to be carried out with a smaller number of people
• It’s more difficult to present results in neat charts and graphs
• The analysis can be more time-consuming and complex

Quantitive and qualitative data in action

A good example of user testing that utilises both quantitive and qualitative data is card-sorting.

In card sorting, a participant has a pile of cards with a different category, subject or article from your website written on each card. They are then asked to group the cards together in a way that seems logical to them – thereby creating a possible set of menus for your website.

Where there is a pattern in the results, the quantitative data can support any decisions you make. However when you need to find out the reasons why people haven’t grouped stuff together or chosen a particular category name, you can then read or listen to their comments to find out their reasonings.

Card-sorting can also be done remotely online, as you can see from the UserZoom example below…

Hmm, this is a much better example than the Radiohead one. Maybe I should have started with that instead.

Well, it’s too late now.

Please note: I originally tried to find a photo of Thom Yorke weighing an amount of things in one hand against the other hand for my header image, but my research proved disappointing.