What’s the Speed Limit? Nonexistent!
Summer has officially started for the Hi, Mileage! team and sprint 7 has been progressing at top speed. We’ve used our momentum and findings from the spring semester to dive right into building, prototyping, and validating. Our focus has been on the test drive as the team explores solutions to enhance the experience.
Let’s Have a Proactive Conversation
A proactive conversational agent engages with a user first whereas a reactive conversational agent only reacts or answers questions. Our goal for the first prototype of the summer was to gauge the level of comfort shoppers have with a proactive conversational agent. We hypothesized that as proactivity increased, the level of comfortability may decrease but the value to shoppers would increase. To test this hypothesis, the team designed a wizard of oz conversational agent that would interact with shoppers during a test drive.
The Levels of Proactivity
We completed both in-person testing with members of our cohort as well as virtual testing with CarMax customers for this first round of testing. The nature of the testing or protocol did not significantly change across in-person and virtual testing. The main focus for both methods was the interactions participants have with the conversational agent. We used a platform called OBS to replicate the environment of a test drive in a virtual setting.
Participants were given a scenario car and asked to complete an approximately 15 minute test drive. The conversational agent was only reactive for the first 5 minutes, engaged with low proactivity for the next 5 minutes, and finally engaged with high proactivity for the last 5 mins. A conversational agent that has low proactivity generally asks for confirmation before providing information whereas a conversational agent with high proactivity simply gives the information using both prior and real-time data.
Visual Screens
For both the in-person and virtual test drive, the conversational experience was accompanied by visual screens. The brief on-boarding screens gave participants a sense of what types of questions could be asked. The following screens, showing the microphone and speaker icon, were used to indicate the status of the conversational agent to users. It provided a visual cue as to when the agent was processing the information and when the agent was finished speaking.
Early Findings
After completing thirteen rounds of testing and synthesis, the team was able to identify key findings and patterns seen across the board. While there were concerns with how effectively a test drive could be replicated virtually, participants responded well and were able to successfully engage with the conversational agent.
General Key Findings:
- People were generally comfortable with an agent that is always on during the test drive as long as they are told that it will be on, and there is an option for them to turn it off.
- There is an expectation that once data is given, it can and will be used in later interactions.
- People did not expect the conversational agent to engage first but the engagement gave them new information and reminded them of forgotten questions.
- People want detailed information about the operation of the car they are test driving.
- Proactivity is appreciated as long as the information is relevant and accurate to the needs of the driver.
Coming Soon!
Moving into sprint 8, our team will continue to test and validate our hypothesis around the test drive experience. We plan to focus specifically on what the team has named the “kicking the tires” process. It refers to the interactions shoppers have right before actually getting into the car for the test drive. We want to see how the experience changes for shoppers when asked to interact with a conversational agent while “kicking the tires”.
