Research blitz with freelance healthcare providers
I recently had the opportunity to plan and lead a research blitz for our product team where we spent three weeks to get to know our users and their needs better than we knew them before.
Our product team was responsible for creating software applications that doctors used to find and do freelance work. In the healthcare industry freelance or contract work is known as “locum tenens.” Locum tenens is defined as a person who temporarily fulfills the duties of another.
Many of us on the team were new to this product space and so in addition to learning more about our users and their needs, we also wanted to do that learning in real-time together. Doing three weeks of interviews together was a great way for our team to create a unified vision and shared understanding going forward.
Our product team
When I talk about our product team I’m referring to four individuals, the product manager (PM), UX researcher, UX designer and senior UX designer.
What were we trying to learn?
Like most product teams we constantly needed to stay close to our users so we could confidently choose the right features and projects to focus on and build. With this research blitz we wanted to answer the following question.
What should we be building next that will provide value to our users?
Our plan
Step 1 — Send out a survey
Step 2 — Interview people who took the survey
The Survey
Everyone on our team already had a bunch of ideas and opinions about what we should build next. So we decided to simply put that list in front of our users and ask them to vote on them. Surveys are great way to objectively measure a bunch of ideas against each other with very little internal company influence on the resulting list.
Using a Qualtrics survey sent by email, we asked our providers to rank from 1 (don’t need) to 10 (must have) the following list of ideas. We also included an open comments field and a checkbox to give us permission to contact them for a follow up interview. The open comment field was really important because we knew it’d be their chance to explain their thoughts and feelings with a little more clarity.
Please note — this is a modified version of the list we sent, I’ve overly simplified this list to keep the details proprietary. The actual survey verbiage was much more descriptive.
- List Jobs
- Notifications
- Signing
- Edit info
- Time entry
- Travel
- History
- Centralize
- Mobile
- Auto-fill
Survey Results
We sent the survey to 4885 providers and we got 153 responses for a 32% response rate.
4885 providers
153 responses
32% response rate
These were the top 3 ideas the providers voted for from the the list:
- Auto-fill
- Centralize
- Signing
Some companies might stop there and say now we know what to build next, or hey, a particular idea won and move on to building it. But we knew that this survey was only the beginning of the process. We knew that the providers taking this survey very likely had different ideas of what each of these ideas actually meant. It was vitally important for us to know what they thought they had voted on.
The Follow-up Interviews
Now that the survey had given us a general idea of what our users were interested in, we needed to ask them to explain in more detail what they wanted. We knew that what we thought they wanted was very likely not what they had in their minds that they wanted.
We had 28 respondents that were willing to be contacted for a follow up interview. We incentivized the providers with gift cards to sweeten the deal. After the scheduling process we had them all booked for an hour each over the course of a three week period. The four of us then cleared our calendars to conduct the interviews.
Interview guidelines
Everyone on the team was willing to put in the time and effort, so we wanted to get the maximum return from the interviews. To get the most out of the interviews we needed to set up some guidelines.
- Product manager and UX designers and researchers attend every interview, no one can skip it or watch it later
- Only one person conducts, but take turns conducting so everyone has a turn to conduct and ask their own questions in their own way
- Meet immediately after every interview to discuss the interview for 30 minutes
- Record everything, both the interview and the post interview discussion
- No interviews on Fridays, take a day off. But do meet on Fridays to review our overall progress and make any tweaks to the process for the following week.
PM and UX attend every interview, no one can skip it or watch it later
It was critical that everyone on the product team attend every interview including the discussions afterwards. Witnessing the same event in real time and then immediately discussing it afterward created the highest possible value return for the group because we processed the experience together while the experience and excitement was still fresh in our minds. Time delays start to decay the experience in our memories and we wanted to avoid that.
Only one person conducts, but take turns conducting so everyone has a turn to conduct and ask their own questions in their own way
Having one person conduct can get to the core information of the interview faster simply because the interviewer and the interviewee are able to spend enough to time to learn to communicate with each other.
We found that the first 5–10 minutes of the interview was where the interviewee and interviewer were mostly just getting familiar with each other. The 10–30 minutes mark was when they started to figure out how to understand each other. And the 30–40 minute mark was when their communication really started to be efficient and productive.
We knew we only wanted one interviewer per interview, but we didn’t want the same person doing all 28 interviews. Each member of our product team had their own perspective and questions. By going round robin, taking turns interviewing, each individual team member had a chance to guide an interview where they personally felt it had the most value. In turn this also gave the other team members a chance to see the new perspective not only in the information revealed but also in the way the other person conducted the interview. In the end, everyone on the team felt like they had had a fair and equal chance to pursue their own ideas and questions.
Meet immediately after every interview to discuss the interview for 30 minutes
Being able to discuss what we witnessed together immediately afterwards created more shared ideas and more shared understanding because the energy and ideas did not have time to fade or fragment. Right after the interview was when everyone’s energy and enthusiasm was at its peak, which in turn also created the ideal conditions for the clearest communication and collaboration for the team.
Looking back on the experience, we all felt that the discussions after the interviews turned out to be one of our favorite parts of the experience. The discussions were fun and exciting because the creativity and energy was very high and we all had a really amazing time sharing in that energy and excitement together.
Another benefit to the discussions after each interview was that they were a chance for the interviewer to get feedback on the interview they had just conducted. During the interview, the interviewer was primarily focused on the interview itself, which meant staying focused and attentive to the interviewee and anticipating where to take the conversation next. We found that a good interviewer does not have time to take notes or get distracted from the conversation or they will end up losing the attention of the interviewee and the conversation will derail. The post interview discussion was the interviewer’s chance to close their mental threads by asking the team questions as well as hearing the feedback and observations from the other team members.
Record everything, both the interview and the post interview discussion
None of us had a photographic memory, and when we were conducting the interviews we would hear more information than our memory could possibly retain. The challenge was that we knew we most likely wouldn’t have a second chance to do this interview again. That meant recording the interview was a must to remember exactly what was said.
We used Zoom to conduct the interviews so we simply turned on Zoom’s record feature to record the interview. We would first ask permission from the interviewee before we turned on the recording, that way they were in control and also build trust between us.
As a side note, none of the people we interviewed declined to be recorded, so don’t ever be afraid to ask. We found that the interviewer understood we needed to remember what was said and understood that what they were saying was really important to us.
No interviews on Fridays, take a day off. But do meet on Fridays to review our overall progress and make any tweaks to the process for the following week
We found that our interview schedule was intense and we all got close to exhaustion by the end of the week. We didn’t do any other company work during the days we were interviewing so we needed at least one work day to catch up on our other work as well as decompress.
Final Interview schedule
Based on the provider’s availability we were able to fit all 28 interviews into a three week timeframe outlined below.
As you can see in the timeframe above, we filled our days with interviews and not much else. As the interviews went by, we also found that four interviews in a day was about the max we could handle and not totally burn ourselves out.
Results and what we learned
As we got through the interviews, some patterns started to emerge in both our interview format and what we were hearing so we adjusted accordingly.
Interview format refinement
At first all four of us had different ways of interviewing. But as we watched each other and talked about each other’s interviews a more simple and refined interview script started to emerge.
Refined interview script
- Introduce who you are and why you are talking to them
- Introduce recording and start recording
- Get to know them, follow conversation threads
- Go through their survey results
- At the end, organically set up talking to them again in the future
What they said they really wanted
As we got near the end of the three weeks there were very clear patterns of ideas emerging that our users were consistently asking for. Probably the most dramatic thing we learned was that the original survey results were only 20% (or less) representative of what our users actually wanted. If we hadn’t actually asked them to explain their survey answers in these one-on-one interviews we would have missed the target.
The original survey results had the following three items at the top of the priority list:
- Auto-fill
- Centralize
- Signing
But as we learned from the interviews, our users wanted centralizing more than anything else, it was their top priority, not number two. In the survey we had written centralizing for just one particular document, but what they told us was that they wanted centralizing for all their documents. And more than that, they wanted all their documents centralized so they could completely skip the submission process, something we hadn’t even considered up to this point. We had only been thinking about documents in isolation rather than in context of the bigger picture.
We also learned that centralizing for our users meant a lot more than just documents, it also meant all their communications. Our users told us in detail about their current communication processes and just how difficult those processes were. We learned about a whole spectrum of communication use cases and problems to solve that our users wanted just as much as the centralized documents. Communication hadn’t even been on our original survey.
In the end, the personal interviews changed everything for us because we finally understood what our users really wanted. The survey ended up being just the first step in the process of discovery. At this point we had accomplished our original goal which was to answer the question, “What should we be building next that will provide value to our users?”
Sharing what we had learned
Now that we had learned what our customers wanted from us we needed to communicate that information to the rest of the software team and stakeholders. We knew that our engineers and stakeholders didn’t want research to just be a black box that supplied answers, they wanted to be brought along the learning journey as well. But they obviously didn’t have time to conduct the surveys and interviews so we had to create a micro journey of what we had just gone through. We decided that a highlights video and a group discussion afterwards was the way to go.
How we created the highlights video
The highlights video needed to be the best quotes of the interviews that showed the overall answers we heard, and cut out everything else. We had 28 hours of of videos to reduce to somewhere around 30 minutes.
Before I knew about the much better tool Dovetail.com, I used a UX research tool called Consider.ly. Because we had recorded all the interviews in Zoom, Zoom had already generated written transcripts of every interview. I took those generated transcripts and uploaded all of them to Consider.ly. Using Consider.ly’s tagging tools I then went methodically through each interview transcript reading and tagging every section of dialog that had seemed worth listening to.
The tags I used emerged organically as I went along, I leaned a lot on the words and terms used during the interviews and our post interview discussions. Over time some tags started to occur more frequently then others. In the end I was able to tag all 28 hours of transcripts with exact timestamps for each quote.
Using Adobe Premiere I then imported all 28 interview recordings into Premiere and then used Premiere’s trimming tools to cut each video down to the timestamps I had identified in the transcripts. I was able to extract the the most important quotes from all 28 hours of interviews into one final 35 minute video ready to show the engineers and the other stakeholders.
Showing the engineers and stakeholders
We put together an hour and a half meeting for the engineers and stakeholders. We prepped them by talking to their leads and managers telling them that we wanted to show a condensed highlights video and discuss it afterwards. Because they knew what we had been doing they were all very excited to have the meeting despite it’s length and having to watch a video.
The video ended up being very powerful because it answered everyone’s questions direct from the source. The discussion afterwards was also very energized and exciting. Many of the engineers starting giving us ideas about things to change, because now they had a lot more clarity on what they were building and why.
The whole experience was very powerful
It’s one thing to read about how to discover what your users really want but it can be life changing when you actually do it. The excitement and energy in our interviews, post interview discussions and with the engineers and stakeholders was way beyond what we imagined. The whole experience was a reminder about why we do what we do. I would highly recommend any product team not only do this exercise at least once, because you’ll never want to live without it again, but plan to do it at least once a year to reset everyone’s focus as the new year begins.
