How do UX professionals select user research methods?

Please note, throughout December we’ll be updating and republishing a few of our favourite articles from last year to give them a new lease of life.

Here’s our former editor Timi Olutu with a UXchat discussing how the pros select their user research methods…

a hand of playing cards

Pick a card… any card. No, this isn’t a card sorting exercise, you UX junkie!

But speaking of UX, card sorting and choices, I find the range of available user research methods fascinating. From the properly scientific to just having a chat with a few people down your local coffee shop.

The Nielsen Norman Group has written an excellent summary of the different research methods available to you… 

user research methods chart

Behavioural –helps uncover how people actually behave when they encounter situations in real life.

Attitudinal –helps uncover how people perceive their own current behaviours and predict future ones.

Qualitative – helps uncover insights about behaviour and attitudes, through direct observation of people (answers the question “Why?”)

Quantitative – helps collect data (often numerical) about behaviours and attitudes, using an instrument and without direct observation of people (answers the questions, “What?” and “How many?”)

In principle, we know which methods to apply in different situations – but what about in practice, when things don’t fit into neat little boxes?

We decided to lob this conundrum at UX professionals who have to regularly perform effective user research, under imperfect circumstances. This article summarises their responses.

This conversation comprised three questions:

  1. Which user research methods do you use, and why?
  2. How do you frame a research question to get the best results?
  3. Given the number of user research methods, is there one in which you’re not 100% a believer?

Which user research methods do you use and why?

First off is Elizabeth Chesters, UX consultant and she’s either a spy or an incredibly dedicated researcher.

“For volunteering projects, I use remote (and international) sessions, guerrilla testing and I’m curious non-stop day to day.”

I identify with Elizabeth’s attitude of “staying curious, non-stop, day to day” but I’m definitely on the ‘spy’ end of the spectrum. What do you think I’m doing when I attend UX events and summarise the learnings? I’m spying on users (or potential users) of our UX testing platform and, of course, UX professionals.

The less people feel like they’re being watched, the more naturally they’re likely to behave. That’s kinda the philosophy behind why our founder put his money on remote unmoderated UX testing.

When you’re always curious, you notice natural behaviours in people, outside a ‘proper’ research context, which you otherwise might not.

Colin, an existential cat, knows what I’m talking about…

I’m more of an absurdist pigeon… but I take Colin’s point about ethnographic lurking”. These studies give you a fly-on-the-wall point of view into users’ long-term habits and behaviours.

You’ll go deeper than usability issues, and understand the problems and lifelong habits that drive people to interact with a system or interface to begin with.

Then you’re equipped to design solutions to human problems, not simply usability issues.

How do you frame a research question to get the best results?

Jackie Mellor Brownlee, a UX professional, sets off an illuminating repartee with Elizabeth Chesters.

Don’t just pen a script and try to force the universe into following your narrative – allow some room for the unexpected and differences in how people process information. If a certain phraseology throws some users off, be willing and prepared to reframe the conversation. 

Read our interview with user research expert Steve Portigal, on preparing for the world to surprise you.

Joe Pendlebury, *THE* UX Chap, reckons there’s a subtle and important art to framing research questions – and he reckons you should leave it to the pros.

“Facilitating a usability testing session, is an art in itself. Leave it to the pros. Too easy to inadvertently ask leading questions.”

Being a non-professional who’s observed multiple forms of user research (at and before WhatUsersDo), I pretty much agree.

I think anyone can learn to write simple usability tasks (as per remote UX testing) but only professionals can tackle the kinds of multifaceted questions that come up during things like diary studies.

For non-UX professionals, it’s soooo easy to ask leading questions because our minds guide us towards definitive answers – especially if they’re answers that validate *your* project.

We (non-professionals) don’t want insights with deeper but less conclusive information – we want a yes or a no. But the problem with leading questions is you’re basically forcing users to give you the oversimplified, incredibly biased answer you already believe – you don’t learn anything.

A nuanced yet accurate answer beats a simple yet innaccurate one, every damn day.

Is there a user research method in which you’re not 100% a believer?

Well, this one really put the existential cat among the absurdist pigeons.

Did unmoderated remote user testing (URUT) make an appearance? Of course not (and we didn’t even bribe anyone) :p

Krystiyan Lukanov, a usability enthusiast, reckons email surveys can be a load of balls though.

“With email surveys and intercept surveys, you can easily get very biased opinions if you don’t follow a very good methodology.”

Depending on where, when and from whom you request surveys, you can end up with a homogenous collection of views.

If you survey people who signed up to your newsletter after buying from your menswear collection, and your survey concerns fashion, I’m guessing you’ll mostly learn how men who like your brand (not people, in general) feel about fashion.

Tom Kerwin, a UX design pro, sums things up nicely, “Use the appropriate tool to solve the current problem.”

Thanks so much for everyone who took part in this #UXchat. Please follow us and tune into Twitter every Thursday at 4pm for more insightful UX based discussion.

Your smartest business move is UX testing.

Try it for yourself – get a free trial showing 3 real people using your website or app, as they speak their thoughts

Main image by Inês Ferreira

Timi is a London-based copywriter and full-time marketing sceptic – there are now more unvalidated opinions out there than ever.

He became a UX testing enthusiast after seeing its power while working at TUI – the world’s largest travel, leisure and tourism company. He then joined WhatUsersDo to sharpen his UX knowledge and work side-by-side with the field’s best and brightest.

Leave a Reply