Takeaways from a Fast-Paced Field Research Session
As user researchers, we all agree that nothing can replace the honest insight provided by the direct observation of a user in his or her own environment when it comes to truly understanding how people act and using that information to build better products. Unfortunately, not many researchers have the luxury to conduct this sort of in-depth field study. My team and I were lucky enough to get a rare opportunity to conduct one across Indonesia’s various islands.
As our first large-scale field-research project conducted within a very tight timeline, we made a lot of mistakes! But this was okay as long as we learned from them — one of the perks of working at a lean company where “failing fast” is a virtue. Against all odds, we managed to execute all phases of the field study process within 4 weeks across 3 cities on various islands! These phases included: Analysis and research planning, execution, and insights delivery.
As such, I wanted to share our learnings, experiences, and advice on how to get these opportunities and make the most of them, in the hope that this will not only set a baseline to help us improve in the future, but you as well!
Part One: Into The Wild
In the second week of May, I was appointed as the lead researcher for a foundational study project. Completely overwhelmed, I had no idea what the goals were or with whom I needed to talk; all I knew was that I was serving one particular stakeholder. However, as a researcher, it knew I needed to do some homework to gather some baseline information before starting any research, let alone a field study, so this is what I did!
I began by asking my stakeholder questions about his goals for the research: what was the purpose and what kinds of insights and recommendations was he hoping to glean. After gathering enough information from him, I developed and presented my research plan to my VP to get feedback and approval to move forward. Finally, with the goals clarified and a plan in place, I began to put together my research team (best team ever!). It consisted of individuals who not only shared the same passion for understanding human behavior, but also were familiar with the characteristics of our target participants.
Below are a few key learnings I took away from going through this initial phase of the process.
1. Involve your stakeholders
In almost every product development project, the following perspectives are often used to understand the problem and the users’ needs, they are:
- Looking in to internal stakeholders
- Looking around to competitors or external inspirations
- Looking out to different types of users
This project was no different. Since we only had one week to finalize our research plan, we wanted to make sure that there would not be any misunderstandings between us and our stakeholder. It would have been a waste of resources if our insights did not answer his questions regarding user behavior. That’s why going to our stakeholder first to really understand the problem he was trying to solve was such an important first step.
Our stakeholder not only provided certain question for us to answer, but we also had an active discussion regarding numerous assumptions about our users. From that meeting, we were able to reach an agreement regarding how we should present our findings and assumptions about user behavior. We believed that each island in Indonesia had unique values that had an impact on the way our users utilize our product. In addition, we decided to divide our storytelling into different segments based on the user journey we developed to make it easier for our stakeholder to empathize with our users. Correspondingly, we came out of these meetings with solid research background, goals, and questions with low rejection rates.
By involving the stakeholder from the beginning, we gave him a sense of ownership in the process which eventually led to a more robust and meaningful outcome since everyone was able to gain a little more perspective and understanding about our users.
2. It’s okay to show your vulnerability
I have always had the belief that a leader should not show any weakness in front of his or her team, no matter how weak he is feeling. Due to this, I have tried to look strong and reliable in front of my team at all times.
Unfortunately, I never been so wrong and this approach backfired spectacularly!
A week after the project was completed, we decided to do an evaluation regarding our performance and how we could improve in the future. From that discussion I realized that my pretense to act strong all the time not only affected my mental health but also increased the anxiety of the rest of my team. Constantly thinking that one member is being more productive than another made us all feel like we were each not contributing enough in this project despite working our asses off! It is amazing how quickly competition, which is supposed to motivate, can turn into burnout when not managed correctly
Those feelings caused us to overwork and neglecting our own health for a month. This led to decreased performance and is certainly not good in the long term. Managing the team and this dynamic was my responsibility as team lead. In the trenches of the day-to-day technical work that is required, it is easy to overlook the well-being of the most important assets of any company: the people who work there. I recommend not forgetting about this and instead investing in your people so they can perform their best for a long time.
Part Two: Surviving The Wild
Often times during this project, we would get back to our hotel room completely exhausted after interviewing 4 to 5 people back-to-back and driving from all over the place. We would be spent and tired, but looking at the big pile of our documentation, notes, and recordings, we knew that we should do a debrief or risk forgetting a lot of good information. However, we’d often succumb to our sleep desires in preparation for doing it all again tomorrow.
This was a mistake!
3. Debrief! Debrief!! Debrief!!!
We know how important it is in user research to do debriefs as quickly as possible after interview sessions with users to ensure key details are retained and the knowledge gained is committed to memory.
In our case, we would often have another participant waiting for us 15–30 minutes away. Since we were doing the interviews at the participant’s house, we planned to use this time while driving to prepare our research tools. On top of this, we buffered an hour extra between each interview. But even this was not enough to handle what we experienced: people love to talk!
This resulted in one of our biggest learnings: buffer enough time for debriefs after each interview by not scheduling more than 3 participants a day and ensuring that the interviews end on time!
As human beings, we often struggle to remember the context of conversations after a certain period of time. Consequently, we unconsciously mix our memory from these conversations with others in an attempt to fill our memory gap. Therefore, in terms of user research, doing debriefs immediately after each interview session is vital to ensuring team members share their main takeaways, make sense of each other’s interpretation methods, and clear misunderstandings.
As a result of our decision to frequently postpone debriefs in the field, we spent a huge amount of time reviewing each participant interview upon returning to the office instead of doing the actual synthesis process and developing our “bigger picture.” This resulted in a lot of lost time and possibly even lost takeaways from each interview.
Part Three: Back From The Wild
We returned back to the office with a huge pile of papers to read and numerous recordings to listen to. We only had 5 days left until D-Day when we needed to present our findings. Yet we were unsure of which research methodology we wanted to use to consolidate and analyze our findings.
There are a number of synthesis methods that one can use depending on the research goals and how one wants to present the insights. One way, for example, is writing down the top insights from each participant interview and see if there are any similar patterns to other users.
However, for our study, since we had around 30 participants and only 5 days so synthesize all this data, we couldn’t use any of the standard methods. Instead, we started dividing our insights based on the user journeys and demographics.
As mentioned earlier, our team and stakeholder already had an agreement about our research framework plan and the hypotheses of cultural impact. Back at the planning stage, we decided to break down our interview and observation guide based on the users’ journey from the moment they open up their store until they close and do recapitulation.
We segregated our guide to ease our participant’s cognitive load to think only one thing at a time and to help him or her easily remember the detail of his behavior. Furthermore, dividing our insights into several touch points throughout the day also made it easier to use as we became more focused each time we discussed specific user journeys.
This method worked well for us because we had assumptions that the users’ demographics were affecting their everyday behavior using our product. It allowed us to quickly analyze the data and synthesize some insights from it.
4. Pick a method ahead of time based on the plan!
The key learning here for us was that we shouldn’t have waited until the last minute to pick a method to synthesize all the data.
We knew that we had a large number of participants to interview, a tight time constraint, and a packed project plan, so we knew that we would have a very limited time to process and analyze the resulting data to make some sort of output for our stakeholder.
Thus, we should have seen this coming. By trying to figure out which method to use with only 5 days remaining, we definitely contributed to our high blood pressure! Use the knowledge of your project timeline and plan to determine the best synthesis methods to use ahead of time.
Conclusion
There may be no such thing as the perfect research; humans are hard to predict and we make mistakes all the time. However, it is our duty as user researchers to learn from these mistakes and try to improve the research process, little by little. Thus, that is the best advice I can give to you from this experience, and my biggest learning:
Acknowledge your mistakes and take steps to do something about them! As long as each research project goes a little better than the last, you are probably doing something right!
Inspired or invigorated by this story? We are actively looking for curious researchers who want to travel around Indonesia, get to know our users, and strive to improve the experiences we provide them! Think you have what it takes? Check out our career site!
Thanks to Yoel for your editing help!
