What is a diary study and why is it useful for longitudinal UX testing?

Continuing our series of beginner’s guides to UX, here’s an overview of diary studies, and their usefulness as long-term usability testing tool.

“Dear Diary, I couldn’t think of anything interesting to write, so here’s some lorem ipsum…”

What is a diary study?

As we stated in our UX dictionary, a diary study gathers together information about a user experience over an extended period of time (or longitudinally as you may hear it described in other places where they like big words).

Participants write about their experiences with a particular product or service in a diary.

They may also take photos or perform other activities to record their experiences. Once the study period is over, the researcher analyses the findings.

Why would you use a diary study?

A diary study is a qualitative research method, so the findings will be based on thoughts, feelings and observable behaviour rather than numerical data.

The longer period of time and the changing context of the user differentiates this method from other tests like surveys or labs, as they only record behaviours based on one specific moment in time, within a single (and unnatural) context.

This is one of the main reasons why we advocate for remote UX testing, as the presence of the person or company running the test can bias the results. It’s difficult to look someone in the eye and tell them their product is flawed.

So, diary studies remove the influence of both the researcher and the unnatural out-of-home setting, but they’re also useful for understanding long-term behaviour.

According to NN/g, these long-term behaviours can include:

  • What time of day do users engage with a product?
  • How do they share content with others?
  • In what capacity do users engage with a product?
  • What are their primary tasks?
  • What are their workflows for completing longer-term tasks?
  • What motivates people to perform specific tasks?
  • How learnable is a system?
  • How loyal are customers over time?
  • How do they perceive a brand after engaging with the corresponding organization?
  • What is the typical customer journey and cross-channel user experience as customers interact with your organization using different devices and channels?
  • What is the cumulative effect of multiple service touchpoints?

All really helpful, and best of all, in-depth and genuinely contextual findings.

How long does a diary study last for?

A diary study can last as long as your project needs. So anywhere from two days to two months. Basically whatever best fits the purpose of the product you’re testing.

“Dear Diary, today I will find out who keeps taking gigantic bites out of my croissants and I will end them, so help me god”

How is a diary study carried out?

During the test period, participants are asked to keep a diary and write down specific information about the activities they’ve been asked to carry out.

UX mastery recommends the user takes photos to explain their activities and highlight things that stood out to them across the course of their day.

According to UPXA, there are three categories by which entries can be collected:

  • Interval-contingent protocol, in which participants have to report their experience at regular predetermined intervals (for example, every two hours or every day).
  • Signal-contingent protocol, which uses a signalling device to prompt the participants to make an entry.
  • Event-contingent protocol, which requires participants to report each time a specific event occurs.

The rate and timing of how people ‘self-report’ should be set up according to research needs. UXPA suggests not being too demanding or the diary will become a “burden to your participants” – a maximum of two to three entries per day should be enough.

The structure of the diary can be open (the tester records entries in their own words) or highly structured (where closed-ended questions are predetermined). It’s up to you how precise you want the information to be.

Are there any special tools you need for a diary study?

There are electronic tools available for people to record their findings in a variety of multimedia ways (text, image, voice recordings), but it’s a good idea to discuss with your participants what method they’d be most comfortable with based on your own testing needs.

UX manager and columnist Peter Hornsby recommended a simple pen and paper approach in his recent AMA:

“I’m not aware of many technology platforms for diary studies; I’ve tended to use just written feedback in whatever format the participant is most comfortable.” Also bear in mind that “if you have to introduce a platform which itself has a learning curve, it can add an additional level of complexity to the process and can colour the client’s experience.”

diary entry

“Dear Diary, nothing interesting to report today so I’ll share with you my quarterly sales figures…”

Disadvantages of diary studies

  • People can be full of good intentions, but then laziness/distractions/Netflix slips in. Hence why it’s possible to set up prompts to notify participants daily to fill in their diary. This however could potentially be quite annoying.
  • The person has to be fully committed to the project. They also need to be comfortable and competent in writing down their findings in clear, legible language and terms.
  • This also means that training sessions for the participants can be lengthy, and you may take a while to whittle down a pool of volunteers.
  • NN/g refers to it as a “poor man’s field study.” A diary study is unlikely to provide observations that are as rich or detailed as a true field study, but they can serve as a decent approximation.
  • Analysing diary studies can also be time-consuming and complex. As we’ll find out below…

How is the research analysed?

Post-study interview: here you can plan a follow-up conversation with the participant and discuss the specific details of the study. You’ll be able to clarify any ambiguous details (illegible handwriting issues, for instance). You can also ask for feedback from the user, which you can use to improve the process for future participants.

Data analysis: NN/g recommends “taking a deep breath” before diving into the vast amount of qualitative data and think about the following questions when evaluating their behaviours: how do they evolve and change over time? What influences these behaviours?

Then you can build a customer journey map to help understand the entire user experience from the perspective of your participant.

Get a newsletter that isn't all about us...

Subscribe for weekly, hand-picked articles about UX, design, and more every Friday—from the Be Good to Your Users blog and the rest of the whole darn web.

...get a little taste right here. 👅 🍭

First diary by Thought Catalog, second diary by Cathryn Lavery, third diary by Raw Pixel.

Christopher Ratcliff

Christopher is the Content Marketing Manager of WhatUsersDo. He’s also a filmmaker and the editor of wayward pop culture site Methods Unsound. He used to be the deputy editor of Econsultancy and editor of Search Engine Watch.

Leave a Reply