Conducting user interviews: case study learnings

Author note: this is a continued exploration of UX processes and methods applied to a specific (fun!) case study. For the background on this, be sure to check out the overview.

Why do user interviews?

In this particular case study, I was fortunate in initial discussions with the stakeholder to find a willingness to invest time to better understand our customers. We knew we had at least 1–2 actual users of this platform, and when I inquired with the stakeholder as to their understanding of the broader industry as well as target users, they had some thoughts in mind.

Here’s some feedback gathered early on through the stakeholder’s prior research:

[They] have employees that are often in the field and use lengthy paper processes. Their work is pretty straightforward but varies. However, all seem to have the same common need for [similar] items. Generally speaking they all immediately get frustrated with a technology when it doesn’t work right off the bat … their resourcefulness to “play around” with something to get it to work is limited. When [technology] is working they are happy and feel very confident.

While it was great to have an idea of who we would be helping, we were lacking more substantial research findings to back those assumptions up. We decided to conduct user interviews, as we felt this would help us better understand our users and the world that our users were living in; we could learn what their real day-to-day was like and understand what true problems and issues users struggled with that perhaps this platform could impact (and what it might not).

Ultimately, any discussion on design would and should be fueled by what we know about our users, not solely by subjective “hey, this would be neat and users would love it!”. By doing these interviews, we’d be better informing our design based on actual user insights. We’d also be keeping ourselves from designing for ourselves (because we, after all, aren’t the actual users!).

Planning user interviews

Aligning Team

Collaboration is key — get the team involved!

Plan your study

Example overview of our original interview study plan outlining objectives

As part of aligning your team, you should set up your study plan. This should include:

• Research objectives: the goal(s) and main purpose of the study
• Target customers: who you’re looking to learn about
• Script and questions: how you’ll prep the participant and what all you’ll be asking them
• Logistics: what you’ll use to run the interviews. This includes:
• Interview location, time(s), duration
• Interview team: who will help you conduct this interview. Example: have a facilitator (lead guiding interview) and one or more notetaker(s) (those who can observe and take notes)
• Tools used (recording devices, in-person vs. remote tools, etc.)
• Participant(s): who you’ve recruited for your study

Note: for your team, be sure to prep the team beforehand! We held a pre-interview meeting to discuss expectations, objectives and make sure we each knew our roles and what we’d need to focus on to make this a good interview.

Gathering assumptions and previous knowledge

Example of some assumed demographics we collected and included in study plan

We included team data and hypotheses early on to include in the study plan. This would help us compare once we had additional interview data to determine what we were findings and how close or far we were from our original ideas.

Determining who to interview
For this, we gathered some assumptions and also expanded who we thought we might want to interview. We wanted to interview not only those using the platform, but also those in related industries or even more abstract fields to see if we might find correlations or large differences to help us understand potential users better.

Recruiting your interviewees

Recruitment channels we highlighted — spoiler alert: we misjudged these!

This turned out to be one of the greatest challenges. Previously, the stakeholder and I had worked across a tech-based products and audiences; recruiting was usually pretty fruitful when using networks like LinkedIn or Facebook.

In this case, we attempted to recruit via similar social channels. These turned out almost no responses at all for this attempt. 😨 😓It would seem, as best we could determine, those we were targeting were not engaged heavily in online social presences, making online outreach far less effective. It was clear we weren’t knowledgable on where to find potential target users we wanted — we did a lot of friends and family-related network recruitment, which didn’t yield us nearly as many participants as we were hoping for.

Running user interviews

I want to share specifically the experience of running our very first interview — this would be with the end customer and user of the platform. It’s one of my favorite interviews to date, simply because we ran into many unexpected issues that yielded some rich learnings as a growing UX researcher.

Our first interview site — held on the back of this truck

Our first interview we wanted to try and get the participant in their natural working environment. We were able to meet them on site and found a place a bit away from the main working areas to conduct our interview. We attempted to preface the interview and set expectations for our purpose: to learn about their day-to-day, not specifically anything around the application they were familiar with (although it was expected to come up).

This was one of the most interesting interview, as we ran into several unexpected aspects:

• Odd location logistics: we ended up interviewing the user on the tail end of a nearby truck (see above), about ~10–15 feet away from the main working area. There wasn’t much room and it wasn’t the most accommodating to insulate from noise or nearby activity. From here, many of the employees would come and go within view throughout the interview — this was a bit distracting from focused conversation.
• Discussion distractions: Despite attempting to set expectations beforehand that this was not a product wishlist discussion session, it was clear that the participant(s) were focused on communicating what they wanted to see in the app. Having both the stakeholder and the stakeholder’s client in the same interview led to natural discussion of future aspirational visions of application capabilities — which, while interesting, wasn’t what we were aiming to get from our allotted 1-1.25 hours.
• Question avoidance: Relatedly, when we attempted to learn about their recollections around how they accomplish and run their work, we’d get replies of “you don’t need to worry about that, the app won’t help us with that”. This derailed from getting a true, honest picture of what all was involved in their day-to-day as they assumed we were only focused on app capabilities vs. just learning generally about them, their work and all related experiences.
• Impromptu participants: We were supposed to be interviewing one person, but a key coworker of our participant noticed us talking and would periodically pop in and join in the discussion. This made it extremely challenging, as you could tell it impacted the natural answers of our original participant and they’d even debate with each other in the midst of discussion.
• Low battery issues: myself, as the facilitator, made a total rookie mistake — we’d planned to record using my mobile phone, which I had not checked for sufficient charge before the interview. 🤦‍♀ Thankfully my fellow team member was able to supplement with their working phone instead. 😅

Subsequent interviews were run from a private location with just the participant, facilitator and notetaker; some participants brought materials and tools (laptops, phones, etc.) that they used on the job to explain their work.

Post-interview activities

After the meetings, we collected any recorded audio and notes and dropped them in Google Drive — this worked great to centralize our materials for any of the team to access.

Note: I’ll admit another miss on my part for this one — I took some notes on paper during an interview and didn’t quickly get them into a centrally stored location after the interview. The paper was misplaced, resulting in lost initial notes. 😭 Learn from my mistake: get those notes safely secured so you don’t lose valuable observations!

Outcomes

Despite key issues such as struggling to find and recruit participants as well as hiccups in the interviewing process, we did get some really interesting insights. We did a mini-workshop to talk through what a persona might look like using these insights.

An initial peek at building our persona used in a later journey map session — meet Tom!

This was very helpful to allow us to always return to “what does Tom think?What do we know about Tom’s experience?” Now we didn’t have to theorize what our users might think, feel, or experience — we had real data from interviews to pull from.

We learned:

• Our users often came from varied backgrounds and landed in their field through on-the-job education
• They spent a lot of time balancing customer sales, job management, team and inventory management — they seemed to feel time was limited and therefore valuable, needing to be put towards whatever helped them get jobs in and done
• They use a variety of physical and/or digital tools — they struggled finding user-friendly experiences that didn’t take a lot of time and training
• They struggled trying to translate manual processes into centrally managed data
• Pain points included finding effective ways to manage jobs, revolving door of employees, and inventory management

Learnings

Here are main takeaways from our interviewing experience:

• Invest in recruitment: we had issues in both quantity of users and finding effective recruitment channels. Be sure to plan your recruitment sources and tactics wisely. Value your participants’ time, through methods such as direct incentives and thank you follow-ups. Dig into where the best source(s) are to tap into communities and networks where potential participants may be. Get a good number of interviews to create a rich opportunity for diverse observations that can help you expand understandings.
• Set expectations (even multiple times!): while we worked to help the participant(s) understand the purpose of our interview sessions, our first interview definitely showed that some folks come in with strong opinions on what they are expecting as well as what they want to discuss and focus on (which may or may not be what you want to cover). Be sure to cover purpose and expectations!
• Get interview-friendly spaces: our first interview gave us an interesting insights into their day-to-day working environment, but it also made it hard to conduct a one-on-one discussion. Scout your intended interview area beforehand. You want to avoid highly distractive areas and/or crowds that cause interruptions.
• Do a test run: give it a practice run-through beforehand. This will help avoid issues such as tech failures, ill-charged devices, and other factors that could impact your ability to conduct an interview and collect your needed takeaways (audio, video, notes, etc.). Have Plan B and even C in place in case your primary tools fall through.
• Prep the team: if we did this again, I would avoid having both the stakeholder and primary client in the same in-person interview as this led to difficulties not getting into “what would we want in the app?” discussions. Plan your team and assign roles — both in-person and, if needed, remote for observation. Make sure the team understands what you’re there for and how to help support the interview process to uncover real experiences.

If you find yourself unsure who you’re designing for — or perhaps the team is making a lot of “I think” statements around the user, focus on conducting real user interviews. Interviews are an immensely valuable tool for helping us understand and uncover real human experiences. These are great to conduct as early as possible in any discovery process and can help rally the team around the user experience and inform the direction of any product design efforts.

Got comments, feedback or questions? Always up to chat UX! You can find me on LinkedIn, Twitter, or Instagram.