Getting Up to Speed on Chatbots
By David Dindi, Isaac Madan, and Anoushka Vaswani
Chatbots have generated a new craze. With major announcements from Google, Facebook, Microsoft, and Amazon, the bot landscape has grown dramatically over the last few months. From shopping to the weather, there increasingly seems to be ‘a bot for that.’ Many questions are yet to be answered and the way in which this ecosystem will evolve in the US is still unknown. Will chat become the dominant OS? Are bots the new apps?
Chatbots are interesting on multiple dimensions, from shifting consumer behavior to technological progress in AI and deep learning. To follow developments in the world of bots and better understand their potential, we’ve outlined some of our favorite resources below. While certainly not comprehensive, there’s a lot here, and we’ll continue to update this list — if there’s something we should add, let us know.
General Resources
Chris Messina’s predictions around the rise of conversational commerce: Link
Dan Grover provides an excellent overview of the evolution of the chat ecosystem and a critique of today’s ‘bot-mania,’ arguing that the chatbot craze stems from the failure of OS makers to adequately serve their users’ needs: Link
Will embedded experiences replace apps and lead to the demise of the app store? Link
Designing chatbots given the current state of AI: Link
Please have your AI talk to my AI: Link
A useful market map of the intelligent assistant landscape: Link
Intercom discusses how bots and humans are good at different things, and outlines cases where a) bots provide a sub-optimal user experience, and b) where humans may be better positioned than bots: Link
Matt Schlicht provides an introduction to the chatbot ecosystem: Link
In this video presentation, Tim Chang of Mayfield discusses definitions of chat interfaces, chatbots, and AI (13:10); interesting questions and considerations (14:55); core competencies for messaging startups (21:30); future trends (23:20); what we will see emerge in the ecosystem (29:15); what investors will be looking for (31:15); and, the value of AI talent and proprietary data (34:15): Link
A list of the “most powerful” chatbots over the course of history: Link
In this comprehensive podcast, Connie Chan, Chris Messina, and Benedict Evans put bots in perspective and discuss chat as an interface: Link
Platform Announcements
Microsoft announces Bot Framework (03/16): Link
David Marcus’ Keynote @ Facebook F8 ’16 (04/16): Link
The Slack Platform Roadmap (04/16): Link
Telegram’s Bot Platform 2.0 (04/16): Link
Viv’s launch at TechCrunch Disrupt (05/16): Link
Technical Resources
Adam Nichol explains that from a technical perspective, there’s still much work to be done to build conversational interfaces and reliance on technology that is not necessarily new. He shares his excitement around technology that enables bots to manage a complex, evolving state and describes current approaches: Link
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing by Kumar et al. (2016). This paper present a neural network architecture (Dynamic Memory Network) for question answering. The network is able to collect memories and to use those memories to answer questions asked to it in natural language. An example is provided below:
Memory 1: Jane went to the hallway.
Memory 2: Mary walked to the bathroom.
Memory 3: Sandra went to the garden.
Memory 4: Daniel went back to the garden.
Memory 5: Sandra took the milk there.
Question: Where is the milk?
Network Answer: garden.
To answer the question posed above, one would need to understand where “there” is by reasoning transitively over Memory 3 and Memory 5. The network is able to reason by transitivity by using an attention mechanism that allows it to focus on — and connect together — the most relevant memories. This network achieves state-of-the-art performance on the Facebook bAbi tasks that test a model’s ability to reason. The Dynamic Memory Network has since been extended to answering questions about images by Xiong et al (2016). An example is provided below — the lighter segments are the sections that the model is focusing on in order to answer the question.
The IBM Watson team presents a new deep learning approach called an Attention Network that provides improved results on question-answering problems (where there’s the need to find an answer in a set of documents): Link
In this tutorial, Igor Khomenko instructs us how to host our own chatbot on AWS: Link
More reading on attention mechanisms:
- Text Understanding with the Attention Sum Reader Network by IBM Watson team (March 2016).
More reading on memory networks:
- Dynamic Memory Networks for Visual and Textual Question Answering by Socher et al. (March 2016).
- Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks by Facebook AI Research (February 2016).
Implementation
Community
Bay Area Bot Chat and Conversational App Developers
Talkabot: Two days of bot talks and community building (September 28–29, Texas).
By David Dindi, Isaac Madan, and Anoushka Vaswani. Anoushka and Isaac are investors at Matrix and Venrock, respectively, and David is a grad student at Stanford and Head TA for CS 224D. If you’re interested in NLP or the conversational ecosystem, we’d love to hear from you.
Requests for Startups is a newsletter of entrepreneurial ideas & perspectives by investors, operators, and influencers. Subscribe to our email newsletter here.
