Applying Participant Experience Principles to Diary Study Design
As UX professionals, we are constantly asking ourselves what it’s like to take part in our studies, how can we make it better, and how do the tools we use affect our participants and our results. We’re acutely aware that how participants feel will have a significant impact on the outcomes and quality of our research.
Most of these concerns and questions relate to participant experience (PX). That is — us as researchers, trying to understand and optimise how participants will engage with our studies. Since we know that it has a significant impact our research, we invest time into planning good PX. This becomes particularly important when we are using methods that are known for being burdensome for participants, such as diary studies.
Diary studies are great for understanding people’s behaviour over time but have notoriously poor PX because they require a lot of effort over an extended period of time. Typically, participants are required to write regular updates over several days or weeks, using an unfamiliar app. Usually participants send these updates to remote researchers, who they haven’t met or spoken to. To the participant, the researcher can appear detached, the tasks sometimes seem irrelevant, and the tools can prove cumbersome to use. As a result, diary studies often suffer from:
· High participant drop-out rate
· Infrequent diary entries
· Entries that lack effort, context and detail
In a recent diary study, we thought we’d face exactly these issues, especially as our research goals meant we needed 60 participants to contribute entries several times a day, for an entire month. We preempted these issues by planning a study with PX at its core. We challenged ourselves to:
Here, we describe how we applied these principles to the design of this diary study and how it influenced the outcomes.
Make it easy
Traditionally, researchers use email or purpose-built software to gather qualitative diary data. Purpose-built tools are great for analysing rich and unstructured data, but we needed a tool that would help us gain an accurate insight into participants’ daily activities.
We had a few key criteria.
· Seamless integration with participants’ existing habits, so the diary felt less intrusive.
· Low friction, so there are no new interfaces to learn or complicated login steps that would get in the way of providing regular diary entries.
· Familiarity, so that we could minimise the time we spent providing technical support and focus communicating with participants in a timely and efficient way.
· Flexibility, we didn’t know what sort of data this exploratory study would yield, so we needed a tool that would adapt to unexpected circumstances.
We considered options ranging from purpose-built tools to email and instant messaging apps. For this study, we chose WhatsApp. The messaging app is popular in the UK (where this study was conducted), so it reduced barriers to uptake and engagement, because our participants used it frequently anyway. There was no new software to learn or long email threads to navigate. Using WhatsApp helped us receive thousands of real-time updates from participants. Over the the month-long study we had just 3 drop outs. The rest remained highly engaged.
The approach we took aligned to our study’s research objectives, but the principles can be applied elsewhere. If you were running a diary study with call centre employees, for example, it would be important to choose a tool that fits the environment they work in and tasks they do. Making it easy isn’t just about the tool. It might, for example, involve identifying the most appropriate time to communicate or even the format of the messages.
Make it human
We wanted to minimise the perception that researchers were monitoring the study from afar as we believed this would reduce engagement. But, with 60 participants in our study, we knew it would be challenging to establish connections with individuals.
Participants in our study received a welcome pack, outlining the tasks and introducing ourselves (including headshot photos). The pack emphasised that we were interested in what they had to say and were looking forward to hearing what was important to them.
We set up a WhatsApp account on a phone that was dedicated to the study. We saved all our participants’ phone numbers into the phone allowing us to send broadcast messages and individual responses. We shared responsibilty for communicating with participants throughout the study.
By using WhatsApp, a place where users already hang out online, we set the tone of the relationship between participant and researcher. We communicated with participants in the language they used themselves in this space. For example, we avoided overly-formal messages like:
“Dear diary study participant, please continue to send us updates over the weekend. Best regards.”
Instead, we wrote:
“Good morning, Thanks for your messages this week, we’ve really enjoyed reading them. Please keep it up over the weekend and we’ll talk soon.”
As a result, participants were forthcoming about their private lives. We gathered rich insights into their activities and needs, accompanied by emojis, pictures and even videos. In some cases, it emerged that participants had added the diary study phone number to their contacts as “Guy & Anya”. For us, this really demonstrated that we had helped humanise the diary study experience.
Make it collaborative
With such a large volume of data coming in, it was tempting to narrow participants’ focus by directing them to share updates on themes of interest. However, this risked undermining a key value: that we were interested in what they had to say. As our research was exploratory, we kept the net wide and filtered out irrelevant ‘noise’ data during our analysis.
However, embracing this level of uncertainty called for an unusually collaborative process with our client stakeholders. Rather than conducting analysis behind closed doors, we invited stakeholders to co-analysis sessions. By encouraging them to step through the (carefully anonymised) data on a regular basis and highlight important entries, we succeeded in pulling out increasingly relevant material in each subsequent round of analysis.
Prior to each co-analysis sessions, stakeholders received a summary of participant stories and emerging themes. In the session itself, the team gathered round a screen and dug deeper into the data. Our stakeholders witnessed participant stories unfold. They empathised with participant needs, learned how to use the data, and became advocates for the methodology itself.
Participant Experience shapes the data
By designing our study with PX in mind we overcame barriers to participant engagement. We collected over 5,000 rich diary entries. We threaded the entries together to build ‘stories’, following tasks, projects, and events as they unfolded. By treating the ‘story’ rather than the ‘entry’ as the basic building block of the diary study, we were able to create coherent narratives that resonated with stakeholders.
It wasn’t just the quantity of entries that were shaped by the PX; the content was too. People spoke to us with a level of intimacy and honesty which went beyond even their friendships. We got real-time insights into their triumphs, frustrations and even relationships.
Apply the principles, but adapt the approach
How can you use these principles to improve the PX of your diary studies? First, cover the basics by establishing research questions, what kind of data you need to answer that question and who your participants should be. Then, start planning your diary study with the principles of easy, human and collaborative.
Make it easy: think about what will enable your participants to generate the data you need, in the environment relevant to your research, with as little effort as possible on their side.
Make it human: consider ways to engage participants, whilst building trust and ensuring you as the researcher stay connected and relatable.
Make it collaborative: seek opportunities to work with participants rather than directing them. Look for ways to bring stakeholders on the journey with you, by building their empathy with participants and helping them understand how to use the research outputs.
The crucial thing is to adapt your approach to the unique needs of your project but, by keeping these principles in mind, we believe researchers can improve PX and therefore the success of your diary studies. You can apply these principles to other research methods too. If you do, we’re interested to hear how you get on.
Authors: Anya Zeitlin, UX Research Consultant at EY-Seren; Guy Simpson, UX Research Consultant at EY-Seren; and David Loughlin, Principal UX Research Consultant at EY-Seren (from left to right)
If you enjoyed this article, feel free to clap 👏 to help others find it.
To learn more about UX research, join our community of over 5000+ UX researchers in the Mixed Methods Slack group 💬
Prefer to listen in? Check out our podcast 🎧
