Driving Conversations Between Buyers and Sellers via Chat
Back in April, our worlds were upside down. With Covid-19 impacting businesses large and small including the small business owners on 1stDibs, our sellers more than ever needed sales leads.
1stDibs is a two-sided marketplace that connects buyers to sellers through our platform. We knew that conversations from interested buyers were a clear and leading indicator of future order conversion. To ensure we were connecting as many potential buyers to sellers, we set out to increase the rate of conversations by developing a chat feature.
Hypothesis
By persisting a chat experience on product details pages, our hypothesis was that we’d remind buyers they could ask questions and start a conversation with the seller. By driving up incremental conversations, we’d also see an incremental lift in orders.
Goals
We set out to develop a chat feature that would:
- Create an easy way for buyers to send messages to sellers
- Send more leads in the form of conversations to sellers
- Increase order volume and GMV
In addition, we wanted to quickly iterate, develop, and test an MVP to determine impact before developing more robust messaging features.
The Feature
On product details pages, once a shopper reached mid-page, we’d trigger a chat box. The chat box prompts the visitor to start a conversation with the seller. The visitor can send a message from the chat box. If they are logged out or unregistered, we prompt them to log in or create an account. Visitors can minimize or dismiss the chat box, and that preference is stored during the duration of their session.
In its initial iteration, the chat box would only support the ability for the prospective buyer to send a message. Seller replies would not appear in the experience because we did not have infrastructure for a true real-time chat experience. If the feature was successful, we’d determine how to support real-time chat in a future iteration.
Risks
There were a few risks we wanted to mitigate. The first was that the feature could just cannibalize contact submits from other touchpoints on the product detail page and not actually drive incremental contacts. Were users abandoning the other conversation touchpoints to submit from chat instead?
Second, we could be sending less qualified messages to sellers. Chat is inherently more casual than a form submission, and we wanted to have a sense for message quality. Did these messages lead to sales at the same rate as messages from other touchpoints?
Lastly, we worried that a chat box triggering on every PDP would be annoying to users. How often were users dismissing the chat box during their session?
We set up new GA tracking and planned to run the feature as an A/B test to answer these questions.
Design and Development
Once we’d outlined our requirements and use cases, designer extraordinaire Eunie Kim quickly developed a design that we shared with engineering and stakeholders. Rob Ianna, our exceptional senior developer, was able to develop the feature in less than a week and we launched the feature to 20% of users with the help of Todd Petty, our fastidious QA engineer. We did all of this in one sprint, from home, and during the height of the pandemic when nothing was certain except the need to be singularly focused on shipping features that would enable sales for our sellers.
Results
We ran the feature as an A/B test for about a month and determined that the chat feature was in fact driving incremental conversations. The variant saw a 20% increase in conversation submits compared to the control. Analysis also showed that these conversations were driving incremental orders.
To gauge message quality, we took a look at word count distribution for chat messages. While many messages were short in word count, they were usually qualified messages.
We rolled the feature out to 100% and continued to see incremental conversation submits as a result of the feature.
Chat V2
Having determined the value of chat via our MVP, we moved forward with developing a real-time chat app that supported the entire buyer:seller conversation. In this version of chat, buyers can send a message, see when the seller is responding in real time via a “seller is typing” indicator, see when new messages are present, and view all messages from the seller including private offers, shipping quotes, and other conversation milestones.
We A/B tested the feature to understand impact and testing showed a 9% increase in overall contact submits driven by a 30% increase in chat app contact submits.
Conclusion
What started out as a scramble during unsettling times wound up being a success story about how teammates can work together to quickly develop something impactful, about the benefits of testing and iterating, and the value of agility, collaboration, and dedication to mission.
