“A Fly on the Wall” — The Power of Field Research
Our team hit the pavement to understand how customers think about insurance, but we learned a lot more than that.
Our UX designers and researchers work closely together to build better customer experiences. Recently, Senior Researchers Mali Aghanasiri and Grace Phang, along with Senior Product Designer John Dale, went out into the field to get a clearer understanding of how users interact with our products.
What is Field Research?
Field research takes UX practitioners out of the usability lab and into the field. By going into the real environment, researchers can observe users acting more naturally than in a typical research lab setting.
“When this method of researching a new mobile web application was proposed, it was really exciting to me. As someone who studied anthropology before coming to design and UX, the idea of participating in an ethnographic study — watching users interact with an application, how they behave while using it, and giving feedback in real-time — was very rewarding. As close as possible to being a fly on the wall.”
John Dale, Senior Product designer
Project Background
The UX Research team at Liberty Mutual recently led a study to gather feedback on a new mobile web tool built by one of our emerging technology groups. The web application was designed to help customers take and upload pictures of their vehicle after an accident to help expedite a claims process. This helps both our customers resolve their claim faster, by getting them involved in the Claims process earlier, and our adjusters — by getting them the right images to create an estimate of the damage to the vehicle more quickly.
Field research methodology was selected for this project as the team not only needed to see how users interact with the app but also wanted to examine the viability of the pictures submitted through this app.
Processes and Steps
When it comes to field research, there’s no overstating the importance of solid prep work. Recruiting the right participants and designing a thoughtful set of questions are critical to the success of the study as a whole.
A strong participant in an ethnographic study is one who is a primary user of the product you’re looking to test. As researchers, it’s on us to set expectations with participants. These types of studies make it necessary for us to get a look at the real life of our participants. We may travel to offices or homes or whatever other environment is necessary to get the most authentic product experience.
For this particular study, we created a screener, worked with a recruiting team, and set very clear expectations with our participants prior to the test days.
This preparation also included creating a moderator guide, deciding on a note-taking methodology, and setting up equipment. All of these steps were necessary to reduce the unknowns and minimize distractions for our participants during the sessions.
Adapting from Pilot to Participants
Prior to running sessions with customers, our research team at Liberty Mutual typically completes a pilot session. The pilot session has proved useful for several reasons:
- Time how long a session will take. Do we have to cut anything out? Can we add anything in?
- Identify problematic areas of the moderator guide. We want to make sure everything is clear and easily understood.
- Ensure that the goals and objectives of the study are met. We want to make sure we’ve prioritized the most important questions.
- Add in other important questions. Sometimes, seeing a participant interact with a product can prompt more interesting lines of follow-up questions.
For this study pilot, five Liberty Mutual employees volunteered to be participants. We set up a sample car (our User Research Director’s car) on top of a parking garage close to the office.
Between the pilot session and the field research sessions two weeks later, we made a few critical changes to our mod guide and technology.
During the pilot session, our team identified a major point of frustration for participants was the slow loading time of the prototype of the product we were using. Customers were focusing heavily on this instead of the actual prototype. To mitigate this, the research team took a closer look at the tools we were using, specifically, a web-based recording tool and a mobile wireless connection. We switched to native screen recording technology, and effectively avoided a stumbling block that could have derailed our study.
Facing the Reality
Our researchers, now with working technology and a solid moderator guide, made their way to our participants’ houses. Before asking participants to use the application, we discussed their experiences with insurance in general, and auto claims in particular. Along with some demographic information, this allowed us to create background profiles to couch our observations.
From here, we could provide scenarios to our participants and ask them to act as though they needed to use our app to file a claim for their car.
After studies like these, we have a high level of confidence in the data we collect precisely because of the methodology we use. Being able to observe users interacting with the application and then asking questions immediately after ensures accuracy as a result of natural behaviors because of the comfortable setting.
Embracing the Unexpected
No matter how prepared you are, there will always be unexpected moments in a field research study. All you can do is be flexible, act quickly, laugh, do your best, and move forward.
We encountered a few of these moments ourselves. Including:
- A police car that stopped by and asked our participant if everything was okay
- A rambunctious Dalmatian dog
- A pouring rainstorm
- Noise from construction and yardwork, and church bells playing “America the Beautiful.”
Big Takeaways
Field research provides the closest analog to the real-world observation of customers using your product. Of course, any time someone is observed and recorded, it affects their behavior. But by working with them in their own environment, around their own car, our team was able to come as close as possible to simulating the natural conditions the app would be used in. This adds to our confidence in the truthfulness and objectivity of our observations and the accuracy of our results.
Preparation and organization is significant. But it’s worth it to get the quality of results that we did. This will always be our main goal, because it’s important to get our products right when we release them. After all, this entire project is about making the lives of our customers easier during stressful times.
