TrunkBot or Not?
The story of how Trunk Club almost drank the bot Kool-Aid.
--
A bit of background
In 2015, Trunk Club released a huge new feature to our product — in-app messaging. We gave our customers the ability to connect with their personal stylists in an easy, low-maintenance way. Since that release, we’ve seen healthy results and we began to wonder if we could expand the messaging interface to other parts of our product’s experience.
This year we set out on a mission to redesign our onboarding experience. Simultaneously — tons of companies and publications were evangelizing the transformative potential of bots in user interfaces. As a product designer, I began to think about the ways Trunk Club could utilize a bot and potentially be a leader in this arena. Given the positive results of our messaging experience and the recent popularity of bots, I hypothesized that using a bot to onboard our users might be the secret sauce to designing an amazing onboarding experience.
In typical designer-fashion, I began spreading my desire to try a bot in our onboarding experience. How innovative we would be if we could pull this off! From a user experience point-of-view, it seemed like an awesome idea to teach our users how to use a core feature of our product (messaging), during the sign-up process. The idea was a hit around the office, and so we began the process of designing a bot prototype.
The Design Approach
Our product design team highly values user testing, and we test almost every new design before development. This has prevented us from releasing potentially dangerous and expensive designs, while also learning more about how our users interact with our product.
Testing a bot, however, was new territory for us. Questions began to arise about the level of prototype fidelity we needed, how to transform a traditional UI into a script, and general testing logistics. As the lead designer on this project, I was slightly overwhelmed for a multitude of reasons. However, my biggest concern was ensuring I could thoroughly think through the UX of the design while also creating a prototype powerful enough to capture the feedback we needed. As a team we decided to break the project up into several digestible phases.
Phase 1 included an ideation session with designers, PMs, and developers. During the session we revisited the findings from our previous onboarding user research sessions. We then took those findings and came up with a list of attributes we felt best exemplified a great onboarding experience for Trunk Club. Then we each sketched out iterations of an onboarding experience and presented them to the group.
Phase 2 I took those sketches and used them as a guiding tool to help flesh out concepts that might resonate well with our users. During this phase I focused completely on the UX of a bot-based onboarding (or what we called “conversational onboarding”). This design took shape in the form of a decision tree. Since our “bot” would supply automated answers based on a user’s input, I needed to craft all the possible routes a user could take and the responses they should get.
Phase 3 was the part I was most nervous about — how to prototype said bot. I’m fortunate enough to be a part of a design team comprised of really talented individuals with robust skill sets. One of our team members is amazing at prototyping through code and I could have easily solicited his help to build out a bot-prototype. However, we were working on a tight timeline and I wanted to challenge myself to see if I could test a bot without using a prototype as robust as a hand-coded one. Then I read Sprint by Jake Knapp, and it hit me.
The book articulated how Slack initially tested their bot concept — one we’ve all come to know and adore — and it wasn’t through code. They simply used a texting interface powered by humans behind the scenes. I then realized that we had built a messaging platform that our stylists were already using every day. I hacked together a route from our home screen to our messenger interface, which would then lead participants through a bot-like experience.
The “bot” was a pre-written script that I would copy and paste into the messaging interface. I gave the participant’s predefined responses that I would answer to as a bot. However a user answered, I had a list of responses at my disposal I pasted into the conversation. Rather than fumble around with the engineering of the prototype, this simple hack allowed me to invest more energy into the actual UX of the design.
The Results
We spent a day testing the bot onboarding design with 6 participants. The test illuminated huge areas of opportunity for our team to run with. One extremely insightful finding was that the bot onboarding felt like it required too much work, when in reality it took half the time our normal onboarding takes. Participants gave feedback that messaging with a bot wasn’t intuitive, which caused them to feel overwhelmed. They expressed desire to fill out their preferences through a more traditional survey.
This highlighted a perfect example of reading between the lines to understand what a user needs versus what they say they want. A quick sign-up process did not increase delightfulness, which was one of our initial hypotheses. Our users needed to talk about their style preferences through a user interface that felt comfortable to them. A bot did not seem to provide a comfortable experience.
To be quite frank, our team felt a bit defeated post-synthesis session. While our goal was to simply learn how a bot-based onboarding would be received by our users, we were all crossing our fingers that this might be the key to an amazing experience.
Does this mean Trunk Club will never try bots in the future? Who knows. We have the luxury of giving our customers a real person to help solve their problems. Maybe we’ll use a bot to help stylists better serve their members. Maybe we’ll use a bot to take care of repetitive tasks for our service team, so they can focus on more complex customer problems. The options are endless.
What was amazing about this test was the rich insights we gained without wasting a minute of our developers time. We tested a very radical design, gained insights, and now have a clear path for our next design iteration. While our bot design failed to meet user’s expectations, our testing won the ultimate battle — the battle of truly understanding what our users need in their onboarding experience.
What’s next for our onboarding design? Well that’s currently in the works and will be released cross-platform to a device near you 😉
Have you tested a bot experience? We’d love to hear what you learned.
