How many pizzas does it take to build a chatbot?
Hackers are hungry
Photo: WFP/Pia Facultad
This week, government, business, academia and civil society leaders will gather at Davos to discuss solutions to the world’s biggest challenges — including how new technologies can be leveraged to solve some of the most serious problems we face. At mVAM, we continue to explore how some of these technologies could be used to help eliminate chronic hunger, malnutrition and food insecurity — most recently looking at how chatbots could help collect important information during a humanitarian response.
Last week, our collaborators at Nielsen — one of the early supporters of mVAM — organized a 24-hour hackathon at the Nielsen Tech Hub in New York City. As part of ongoing efforts through Nielsen Cares, the hackathon aimed to develop an open-source humanitarian chatbot that can collect real-time information about food security. This came at the right time for WFP — we’d developed and tested a prototype of the chatbot with InSTEDD, and Nielsen’s technology and development input helped bring in important new capabilities. Ultimately, our goal is to field-test a chatbot in Haiti in the next few months to help us track food security conditions as people recover from the impacts of Hurricane Matthew.
The event was open to the public. A diverse group of students, volunteer hackers, and Nielsen staff showed up to take on the challenge, despite the wintry weather. InSTEDD’s Director of Platform Engineering, Nicolás di Tada also participated.
Much more than a chatbot
What the hackers built is much more that a chatbot: it is a bona-fide chat-based data collection and reporting system. Rather than attempt to outdo each other (as is the case in most hackathons), the teams split up to build the different components of the system. The different teams, made up of perfect strangers, communicated during the hackathon through Slack. After 24 hours, most components were fully coded up, but there were still bugs with the orchestrator and the gateway that additional post-hackathon work will resolve.
The architecture of the system, as defined by Nielsen, includes:
- a gateway that is able to interact with respondents through Facebook Messenger and potentially other chat applications. The Facebook gateway was built on top of the AWS Lambda service;
- a natural language processing engine that analyzes text on the fly. It allows the chatbot to ‘interpret’ a user’s answers. For now, the NLP engine processes English language text, although the engine includes a translation service and, by default, translates all languages to English for more advanced NLP tasks. The engine was built using the AWS Lambda service and leverages IBM Watson’s AlchemyLanguage service for text processing.;
- a set of ‘backend APIs’ that manage respondent and survey data, route respondents from each response to the next question, and provide data to user interfaces . The APIs were built using the Django framework for python and deploys on the AWS Elastic Beanstalk service;
- an ‘orchestration layer’ that maintains survey status and routes messages between the end user and the various backend services. The orchestration service is built on top of the AWS Lambda service; and
Leveraging cloud services from the Amazon Web Services product catalog, the teams were able to build a scalable, cost effective platform that can be deployed quickly to multiple locations globally.
Remember the humans
We also received tips from a chatbot specialist, Alec Lazarescu from Chatbots Magazine. He encouraged us to ‘onboard’ users with an initial message that gives people a clear idea of what the chatbot is for. He told us to avoid ‘dead ends’ and allow users to speak to a human being in case they get stuck.
We’re very grateful to Nielsen for their support and to all the participants for their energy and creativity. The next steps involve WFP and InSTEDD accessing the code and work on ironing out the kinks. We expect challenges with the natural language processing in Haitian Creole, a language that is probably under-researched. Making the different parts of the chatbot work together seamlessly also appears to be an area we will still have to work on. And, of course, the final test will be to see whether our target group — people living in Haiti — find the chatbot engaging.
Originally published at MVAM: THE BLOG.