How I got insight into the needs of Knowcast users using diary studies
We started building prototypes that addressed needs from our users with strong conviction. In this post I show how we did this.
In my previous posts I shared why behaviour-based research is critical and how I used LinkedIn polls to find qualified diary study participants.
How I ran the diary study with WhatsApp and other tools
- Qualify: get green light on contract, agree on payment terms and start date (cohort). Initial video call, 15–20 minutes. One key element here was to check “Will the two weeks we’ve scheduled be a normal period for you?” This was critical as for instance one person was about to go on holiday, where their audio consumption habits were dramatically different. This helped us reschedule to another cohort date that was more representative of their day to day lives.
- Commit: Draft up electronic contract and send for review and digital signature.
- Schedule: send link to guide and invite to bootstrapping.
- Execute: ensure participants are meeting contribution needs (e.g. 5+ listening sessions for both weeks)
- Catalogue & Analyse: Collect videos, transcribe, and add metadata (e.g. location, type of phone, nature of diary entry etc)
The diary study schedule: two weeks with pre- and post-study top and tail calls.
As part of onboarding I sent a link to a comprehensive UX study guide with accompanying cheat sheet. This set expectations about we were expecting participants to do.
- Prior to start: Interview 1: onboarding, sharing past experiences with podcasting
- Week 1 (days 1–7) 5+ Listening Sessions
- Week 2 (days 8–14): Interview 2: mid-study reflection, 5+ Listening Sessions
- After Week 2: Interview 3: post-study reflection
Key concepts: listening sessions and moments that matter
Listening session: users listening to podcasts as normal in their lives
We used the concept of a “listening session” to describe where and when a diary participant — in their everyday lives — listened to a podcast. We set expectations that participants aim do this five or more times a week. Given we qualified with this at multiple levels, this should be fine.
Listening sessions would start by setting the scene with answers to these questions:
- Where are you and where did you just come from?
- What’s going on in your mind?
- What equipment are you using?
- What are you going to listen to, when did you choose it, and why that one?
Moments that matter: users having an “aha!” moment when in a listening session.
Moments that matter were a core foundation to our insights. We were interested in:
- How often they happen
- What the moments were about
- The shelf-life of the moment. E.g. did they care about that moment a week later (hence the value of having at least two weeks in the study).
Guiding questions were:
- Short recap of the moment
- Why does this moment in the episode matter to you?
- What would you like to do about it?
Reflection
At the end of a listening session we asked participants to take a moment to reflect on their experience. Remember, this is their current behaviour, using their current tools, doing everything just as they did before.
Questions we asked them to answer were:
- How did you feel overall?
- How many moments mattered to you in that session?
- What you’d like to do as a result of the overall listening session?
Media collected
We asked them to share:
- Video clips of where they were and what they were doing, in particular of their faces so we could capture body language and emotions
- Screen shots of the podcast episode they were listening to
Tools I used for planning
- Pandadocs for digital contracts.
- Fireflies.ai for transcribing the Google Meet calls
- X.ai (then calendly) for scheduling appointments
Next: how the diary studies cemented our conviction in Knowcast
