What’s the difference between moderated and unmoderated testing?

Welcome to our Ask WhatUsersDo video series, where our very own UX Research team answers your most pressing user research and testing questions, and you can see what we really look like in person.

Just in case you thought we really were hand-drawn, floating heads screaming into the void.

This week, Hazel Ho, Senior UX Researcher, reveals the differences between moderated and unmoderated testing. Check out the 4 minute video below, or skip down to the written transcript.

Video Thumbnail

What’s the Difference Between Moderated and Unmoderated Tests?

What’s the differences between moderated and unmoderated tests?

Generally we do most of our testing using unmoderated sessions. These are great for when tasks are straightforward and the testing is done on a fairly functional website.

Unmoderated is also good for when testing usability on a natural end-to-end journey (i.e. when users are coming to your website then going on a full purchase journey). In these cases, unmoderated tests are sufficient. They’re a really easy way to generate quick results.

But what if things are not so straightforward? In this case, moderated sessions may be better. Moderated sessions are what we do when testing more complex websites or testing a prototype that’s incomplete.

What are the pros of moderated tests?

We have more control over what direction to take the test. If we’re doing something remote unmoderated then we’re just asking questions ’straight-up’ about what’s in front of the user. Whereas in moderated sessions we can probe a little more, we can ask users more ‘why’ questions. So if they’re stuck at a certain stage we can then ask them what they expected to happen next or what the issue is.

If we’re testing on a staging site or a prototype that isn’t fully functional, moderated sessions are really good because we can afford to ask things that are more in-depth and get that valuable insight that we wouldn’t normally get with an unmoderated session.

What are the cons of moderated tests?

We can experience the ‘observer effect’ – this happens when users might act differently then when they were doing something on their own because someone like myself (a user researcher) is in their presence. So when a moderator is in a session with the user, they might feel compelled to give a specific answer or the ‘right’ answer.

You might also run the risk of the moderator asking a question in a way that might lead or bias them to answer a question in an unnatural way.

It’s also more expensive and time consuming, just because we need to factor in scheduling users to participate in sessions that are at a suitable time for both of us.

It’s also worth keeping in mind that you could go crazy and do two types of testing – both moderated and unmoderated sessions.

These tests can work complimentary and it’s something that I often do when clients are testing a brand new website. So we can do unmoderated testing to uncover the main usability issues and then follow up later with moderated sessions, where we can then focus and get further insights on the issues that we uncovered in the first round of testing.

There isn’t really a ‘one-size fits all’ for usability testing. Different stages of websites can benefit from either moderated and unmoderated, or both!

If you have any questions about which method is more suitable, you know who to contact!

Our UX blog is on the move...

We're now publishing all of our brand new content on the UserZoom UX blog. All of our previously published articles will also be migrating to UserZoom over the coming months.

Don't worry, we'll still be just as accessible, interesting, helpful and entertaning as ever. We just have a different name and an owl instead of a question mark for a logo.

Come say hello!

Christopher Ratcliff

Christopher is the Content Marketing Manager of WhatUsersDo. He’s also a filmmaker and the editor of wayward pop culture site Methods Unsound. He used to be the deputy editor of Econsultancy and editor of Search Engine Watch.

Leave a Reply