What’s the difference between quantitative and qualitative research? What is quantitative and qualitative data? What the heck does this have to do with Radiohead?

keys with numbers

In my first few weeks of working for a remote UX testing company (twist: this one) I was put under a rigorous training programme.

Every facet of UX testing was explained to me in detail, with practical examples and demos delivered along with a reading list of all the key UX texts. I’m not saying any of this as some kind of unsubtle recruitment drive (I found the constant running up steps and punching frozen meat to be both humiliating and confusing) but to reveal where I am on my beginner’s journey into UX.

For you see, when discussing the virtues of remote UX testing over other types of testing, two phrases came up regularly that I could only nod sanguinely at, then make a mental note to look up the meaning of later…  quantitative and qualitative research!

SMASH CUT: It’s two months later and I’m only getting around to it now.

Both terms are as hard to pronounce as they are to spell – which is why you’ll often hear UX practitioners refer to them as qual and quant. They’ll tell you it’s to help save their precious time, but they’ll be lying.

Let’s get started…

What is quantitative research?

This is any kind of investigation, experiment or study where the results can be presented with numerical values. The data you’ll uncover in quantitative research is all to do with ‘how many, how often and/or how much, etc.’

For example, let’s say you’ve gathered 30 people together in a room and you’ve asked them all, “What is your favourite Radiohead album?” You then write down the results and have a tally that may look like this:

  • OK Computer: 8
  • Kid A: 6
  • The Bends: 4
  • In Rainbows: 4
  • A Moon Shaped Pool: 3
  • Amnesiac: 2
  • Hail to the Thief: 1
  • The King of Limbs: 1
  • Pablo Honey: 1 (participant expelled from group for being a ‘contrarian and/or joker’)

This is quantitive research and its resulting data in its simplest form.

Why you went to the trouble and expense of gathering people 30 people in a room just to ask them about their favourite Radiohead album on company time… well that’s something you’ll have to justify to your line manager.

*People don’t have to be physically in a space to get quantitative data – you can just do it online
**Please appreciate the fact that I magnanimously chose an objective ‘likely real world’ score for each album rather than putting my actual favourite as the most popular
***As my spellcheck can attest, I can spell magnanimous as well as I can spell quantitative
****Alright, seeing as you asked, it’s Kid A

Of course this is a simplification. For more complex examples of quantitive data just take a look inside the Google Analytics of your website – pageviews, sessions, bounce-rate, frequency of visits over time. This is all quantitative data.

For more complex quantitative research projects, you may have to use an Excel spreadsheet to help reveal the patterns in your data. i.e. How many people who say that The King of Limbs is their favourite Radiohead album also haven’t heard any other Radiohead albums?

Pros of quantitative research

  • Quantitative data is relatively easy to gather, doesn’t necessarily take very long and you’ll find it easier to assemble a larger group of respondents.
  • Fast results
  • Quantitive research tends to be objective (i.e. it’s harder to ‘lead’ participants with more complex questions)
  • It’s easy to make nice visualisations from the raw data
  • People love statistics!

Cons of quantitative research

  • A large sample size is needed for any kind of statistical significance. It’s no good asking only 3 people their favourite Radiohead album, when there are 6 more albums than people who you’re asking.
  • Quantitive data can tell you what is happening and how many times – it can’t tell you WHY it’s happening.
  • If you want to use quantitive data to make changes or improve your product or website, you’ll only be making a guess as to what those changes need to be.

This is where qualitative research comes in…

What is qualitative research?

This is any kind of investigation, experiment or study where the results aren’t in numerical form. These will be the observations, comments, thoughts and feelings of the participants. You may also hear this referred to as empirical research.

Qualitative research can take the form of conversations, interviews, open-ended questionnaires or focus groups.

For example, let’s say you’ve gathered another 30 people together in a room and you’ve asked them all, “What is your favourite Radiohead album and why?”

You will then have a similar quantitive data as the first experiment, but now with extra insight!

  • You may discover that although eight people loved OK Computer, half of them only like it because it’s the album they heard the most.
  • One person might say something about it “perfectly capturing the air of scepticism around the 1997 general election.”
  • Someone might appreciate it for the “multilayered arpeggiated guitars and experimental time signatures.”
  • Someone else may just shrug their shoulders and make a farty noise with their mouth.
  • Either way, you’ve learned something about your participants, the topic of your study and their relationship to one another.

Also, congratulations, you’ve just had 60 people come into your office, specifically to talk to you about Radiohead.

At WhatUsersDo we mainly deal with qualitative research with our remote user testing platform. If you run a UX test on your website using our panel of testers, you’ll receive videos of those tests in which you can see the user interact with your site and, crucially, you’ll hear their thoughts and feelings spoken out loud as they navigate.

Pros of qualitative research

  • You’ll be inside the mind of the person using your product
  • You should hopefully see things that quantitive data can’t reveal – for instance if there’s a page on your website losing traffic, you’ll be able to witness first hand what happens when a genuine user visits the page
  • Participants may find it easier to reveal their feelings about something, rather than assigning a number or ticking a ‘yes or no’ box.
  • This anecdotal evidence can be more persuasive than hard data – especially if it’s emotionally driven and your stakeholders can see and hear these observations for themselves.

Cons of qualitative research

  • Because of the time and resources involved, qualitative research tends to be carried out with a smaller number of people
  • It’s more difficult to present results in neat charts and graphs
  • The analysis can be more time-consuming and complex

Quantitive and qualitative data in action

A good example of user testing that utilises both quantitive and qualitative data is card-sorting.

Example from Optimal Workshop

Example from Optimal Workshop

We discussed this in detail last week, but basically a participant has a pile of cards with a different category, subject or article from your website written on each card. They are then asked to group the cards together in a way that seems logical to them – thereby creating a possible set of menus for your website.

Where there is a pattern in the results, the quantitative data can support any decisions you make. However when you need to find out the reasons why people haven’t grouped stuff together or chosen a particular category name, you can then read or listen to their comments to find out their reasonings.

Hmm, this is a much better example than the Radiohead one. Maybe I should have started with that instead.

Well, it’s too late now.

Get a newsletter that isn't all about us...

Subscribe for weekly, hand-picked articles about UX, design, and more every Friday—from the Be Good to Your Users blog and the rest of the whole darn web.

...get a little taste right here. 👅 🍭

Main image credit: Andre Branch.

I originally tried to find a photo of Thom Yorke weighing an amount of things in one hand against the other hand, but the my research proved disappointing.

Christopher Ratcliff
Christopher is the Content Marketing Manager of WhatUsersDo. He's also a filmmaker and the editor of wayward pop culture site Methods Unsound. He used to be the deputy editor of Econsultancy and editor of Search Engine Watch.

Leave a Reply