Let's get this Conceptual Prototype on the Road!
Flying down the highway at top speed, radio turned all the way up, with the spring presentation circled on our map, sprint 5 is well underway. This sprint is all about validating problem statements and our ideas about how we could use conversation to address them. All this validation is happening right as we are gearing up for the spring presentation, which will be no Sunday drive.
Carceptual Prototype (sorry)
What Needs Validation?
Our goal for the conceptual prototype was to validate the customer needs that we believe we uncovered in our earlier rounds of primary research interviews. We also needed a gut check on how our initial conversational solutions sounded to potential users.
Qualitative Validation
To gather as much rich data as possible, one half of our validation effort was to present a series of 6 storyboards to participants who had recently been to a CarMax in person. This method, referred to as speed dating, is the process of quickly presenting multiple scenarios to a potential user to gather quick feedback on user needs and possible solutions. We wanted initial reactions to both our needs statements and conversational solutions to help us narrow the focus of our project. After getting initial reactions, we probed to discover users’ comfortability with the conversational agent. We felt we were lacking data on how people felt about the capabilities of conversational agents and virtual assistants.
After completing 10 speed dating interviews our team was pleasantly surprised by how smoothly the interviews went. The team was a bit worried about people thinking the method was strange or the scenarios too abstract or confusing, but people responded well to the method and seemed to enjoy giving their reactions. Here are some of our initial findings:
- People were interested in a conversational agent that could help facilitate their own research rather than just feed them data.
- Interacting with a conversational agent can be less stressful than when interacting with a sales consultant.
- People varied in whether they found it helpful to have a sales consultant present when interacting with a conversational agent.
- It is often difficult to understand how to interact with a conversational agent in a unique context, like a test drive.
- The context of interaction with the conversational agent impacts people’s comfortability with sharing their conversational data.
- People tended to trust the information provided by the conversational agent more when it was objective or when a sales consultant was present to contextualize it.
- Receiving recommendations on cars that fit particular needs would be more helpful than maintaining a list of liked cars.
More Validation Please
After consulting our awesome faculty and CarMax sponsors, we decided to add a bit more to the conceptual prototype. In addition to our interviews, we added a large survey to supplement the contextual data with quantitative data.
We created a survey on Qualtrics that asked questions concerning the scenarios presented to the speed dating participants. The survey was administered with the help of CarMax on their website. Within the 2 hours the survey was live, we received 142 respondents. Each respondent was given 2 scenarios and 6 to 7 of the 19 total questions. These questions were posed on a Likert Scale rather than as open-ended responses.
Our team was overjoyed by the volume of responses we received in the brief amount of time that the survey was live. We are still in the process of analyzing the data and applying it to our qualitative findings, but below are just some of our results.
- 60% of respondents found it easy to maintain their own list of liked cars.
- 30% of customers felt more confident when being guided through CarMax’s unique car-buying process.
- 55% of respondents felt it was necessary to have their questions answered immediately during a test drive.
- 69% of respondents were comfortable sharing their conversational data with a sales consultant.
- 96% of respondents felt they had to be at least very confident in their decision before purchasing a car.
- 48% of respondents felt that social factors played a role in their purchase decision.
- 64% of respondents were comfortable asking a car questions.
What’s Up Next?
Narrowing In
With some validation in hand, our team is starting to narrow even further. We are looking forward to exploring some of the conversational solutions from our conceptual prototype. We are planning on conducting more speed dating sessions to test out more specific ideas for a conversational agent in the scenarios we already created.
Spring Presentation
We also have begun planning for the much-hyped spring presentation. We are currently exploring ways to make this as an engaging production as possible given the remote setting. Whatever we come up with, you won’t want to miss it. See you all there!
