Using Chatbots to Provide Faster COVID-19 Community Support

Helpbot is a customer support chatbot powered by machine learning, which replaces the contact flow for guests trying to message Airbnb for support. We used Helpbot to provide intelligent help regarding COVID-19 questions, and developed workflows for our Extenuating Circumstances policy. Over 50% of users engaged with the COVID specific flow, which resulted in a significant number of customers being able to solve their problem instantly.

Guests rely on Airbnb every day to assist them through all stages of a trip. From finding a place to stay, all the way to checking out, and returning home safely.

At the same time, users on Airbnb can have a wide variety of different issues, such as changing their payment methods, or finding their host’s phone number. This makes it essential to build a flexible interface for users to easily find solutions we have available, and, if necessary, get help from an agent. Chatbots have become increasingly popular for this use case.

For this, we created Atis: a chatbot development platform that provides an intuitive and powerful API for building chatbots.

Atis is Airbnb’s intelligent interaction and assistant platform, the ‘brain’ powering Airbnb’s customer support chatbot: Helpbot.

Helpbot is the second step in contacting a human customer support agent. The goal of Helpbot is to quickly understand the users issue, and attempt to solve the problem in an automated way. Helpbot assists guests with many different types of issues: canceling a listing, changing reservation dates, updating notification settings, etc.

Here is a video of a standard Helpbot dialogue that helps a user turn off notifications.

Providing quick, reliable support became even more crucial as we faced the pandemic of COVID-19. During the height of the pandemic, we quickly built certain features into Helpbot to deliver immediate help and support to our guests and hosts who contacted us. This was made possible through the fact that:

1. Chatbots do a better job of understanding the issue a user has by pulling in the user’s context.
2. It’s easier to change the behavior of a chatbot, since every interaction is just treated as a message.

COVID-19 Response

Helpbot was leaned on heavily during the COVID-19 pandemic to help guests cancel reservations, and provide clarity, in accordance with our extenuating circumstances policy. This is a case study on how easily new behavior could be added to chatbots with significant user impact. The following will discuss a few changes we made to Helpbot to support the COVID-19 efforts.

Suggested issues

Helpbot leverages Atis’s NLU model to predict an issue when a user first enters the Helpbot thread. During the COVID-19 crisis, we retrained this model to predict whether or not a user is likely to have a COVID-19 issue, based on features including:

• Recently viewed help center articles
• If they have an active reservation
• If they are a guest or host on the platform

If our model determines that they do, we automatically suggest this as the first prompt. Over 50% of guests answered “yes” to this prompt.

If a user says “yes” to this prompt, then we can immediately understand their issue without requiring the guest to type anything. After this, we deliver a “contextual instant answer” for COVID-19.

Contextual instant answers

COVID-19 affected Airbnb users differently, depending on whether they are a guest or host, whether or not they have a reservation, and when that reservation was booked. Helpbot delivers a different COVID-19 instant answer depending on the status of the user’s account. For instance:

• A user who has a reservation that qualifies for an extenuating circumstances is shown a cancellation flow that help them quickly cancel and get a refund.
• A user who has a reservation that does not qualify for an extenuating circumstance is shown a standard cancellation flow, with refunds that follow the cancellation policy of the listing.

Impact

Overall, helpbot significantly reduced the ticket creation rate, relative to the existing contact flow during the COVID-19 crisis, and successfully supported thousands of hosts and guests.

Atis

Atis is our in-house chatbot platform that Helpbot was developed with. Much of this post will be looking at the internals of how Atis works.

Open vs Guided Chatbot

Most chatbots in academia are open domain chatbots, which allow a user to say anything, and the bot will reply with generated text that is semantically and grammatically correct. However, most chatbots deployed in practice are usually more closed domain, multi-turn, goal oriented chatbots.

The bot triages the customer’s issue with multiple questions or “turns” of dialogue.

Programming Interface

Atis provides a programming interface that allows product teams to quickly develop chatbots. Here is a simple example of building a “hello world”-like single turn chatbot:

This would create a simple bot that asks for your name, waits for a response, and replies: “Hi <name>, it’s great to meet you!”

The Atis dialogue manager knows how to execute any task that implements the TaskHandler class. In this simple task, the dialogue manager will:

• Execute prompt
• Send the message back to the user (“Hi, what is your name?”)
• On response, execute response with the message the user sent back

This code defines a chatbot with 3 turns of dialogue, but does not specify in what order they should be executed. For that, ATIS defines a “domain definition” file, which lays out the order of the turns. The final Weather task executes a skip action. This informs the dialogue manager that this task should not wait for a response, since the bot dialogue is complete.

This definition shows 3 tasks that need to be executed: name_question, location_question, and weather. The initial_task_name defines which task should be performed first, and the next_states defines the next task that should be executed. Atis will automatically load the task handlers by the camel cased name specified in the multi_turn_domain.json file and execute them in the specified order.

Task Handler Lifecycle

Task handlers implement a lifecycle that represents the request / response nature of a typical chatbot conversation.

Much like lifecycles found in Android activity lifecycles, or React lifecycles, Atis tasks also support a lifecycle pattern where tasks are executed by a dialogue manager. The lifecycle methods Atis tasks implement are:

• activate?: This determines whether a task should activate based on the state of a dialogue
• prompt: A prompt to the user, for instance: showing a message or set of options
• response(dialogue_input: UserInputMessage): A response to a prompt, for instance: the message a user typed in, or an option a user selected.

Branching & Activations

The previous example defined a linear dialogue structure, starting with the name question, and ending with weather. Atis can also define tree or graph structured dialogues with activations and edge handlers.

Linear dialogues vs Tree structured dialogues

Here is an example of using task activations to build a tree structured dialogue:

The tasks that are in the branch need to implement an activate? method. The dialogue manager will call both TravelSuggestionAnswer#activate? and WeatherAnswer#activate? to determine which task will be called next.

While the example above is somewhat contrived, this is a powerful framework for expressing complex behaviors.

Edge Handlers

In the examples specified, the activate? method is defined on a TaksHandler. Atis also provides an interface to declare EdgeHandler's, which only implement the activate? method, and are used for when transitions are too complex to squeeze into a single activate? method. For instance, in the example above, the tasks also be written with edge handlers like this:

Like task handlers, the edge handlers will be automatically loaded and evaluated by looking up the camel cased name of the tasks in the edge.

Messages & Actions

Atis supports a wide variety of messages for example:

• Simple message

• Options

• Reservation Card

• Reservation Picker

• Help articles

• … And many more

These can be called in the task handlers to be sent back to the messaging client. The Atis API also supports a set of “actions” that essentially act as control flow directions for the dialogue manager. For instance:

• skip: Skip the response block of a task.
• goto(next_task_name: String): Jump to a task within a workflow.
• transfer(next_workflow_name: String): Transfer to a different workflow.

Note: Workflows are not covered in this post for the sake of brevity. At a high level, they are a way to group a set of tasks together. For example, a cancellation workflow would involve 2 tasks (select reservation → are you sure? → confirmation).

Hooks

Atis offers before and after hooks for lifecycle methods like prompt, response and activate, similar to Rails.

Context Switching

We can combine hooks together with object oriented programming principles to build highly intelligent context switching functionality. Context switching is necessary when the guest expresses an intent out of band. For instance, if a dialogue proceeds like:

In this example, the guest responded with a request instead of their name, and our framework is able to change its behavior and respond with a reservation prompt instead of the standard response of “Hello {name}, it’s great to meet you!”. This is called a context switch.

This can be implemented by building a simple abstract task handler:

This will automatically execute the GreetingTask, if the term “reservation” is found in the message input.

Dialogue Manager

A dialogue manager in a conversational bot is responsible for traversing the dialogue graph. The dialogue manager will decide which tasks to execute by calling the activate? method on task handlers and edge handlers. This allows us to define extremely complex dialogues, like the one we currently use for Helpbot. With the API described above, we are able to build complex flows that walk a guest or host through discovering a solution to their issue, or getting them to the right customer support agent.

Understanding Customer Intents

ATIS comes with a default with a powerful text classifier called WideText: a low latency, CNN-based text classifier.

To motivate the need for such a model, imagine you want to build a chatbot that is able to understand what a user says “I want to cancel my reservation” or “my listing is not very clean” and respond intelligently. One way this can be done is by constructing a taxonomy of issues and predicting among them.

“I want to cancel my reservation” →reservation.guest.cancellation

“My listing is not very clean” →reservation.guest.cleanliness

Typically, a text classification problem is framed as the following:

However, you will realize that other features will be important for understanding the user. For instance, “this house is not clean” could be a reservation.guest.cleanliness if the user is a guest, or reservation.host.cleanliness if the user is a host.

WideText is a PyTorch library that makes it easy to train and deploy multi-modal text classification models.

The exact details of this classifier will be discussed in a separate, future blog post. WideText forms the backbone of our chatbot intelligence.

Circling back to the WhatCanIHelpWith task, here is an example of how WideText can be used to make a prediction:

In this example, we relied on a trained WideText model to understand user intents. The details of how to train and deploy this model will be covered separately.

Helpbot

Helpbot is the “flagship” chatbot that is built on the Atis platform. It handles by far the most messages. At the time of this writing, Helpbot is currently processing over 100k messages per day.

The core purpose of Helpbot is delivering contextual self solve content. When a guest tries to contact Helpbot, we identify their issue by asking the guest a set of questions and leveraging WideText to predict a topic. Then, we present a user with self solve content that could resolve the users issue immediately.

Mapping user issues to a fixed set of ticket topics makes it easy for us to quickly add new types of self solve content like help articles.

Conclusion

Airbnb strives to deliver immediate help to guests and hosts who need it, especially during events like COVID. We leveraged Helpbot’s ability to quickly understand customer issues, and provide self solve solutions to help those affected by the COVID epidemic.

Contributors

Building Atis and integrating it into Helpbot was a multi-team effort. Thank you to everyone that helped make this possible:

Carter Appleton
Stephanie Pang
Jack Chen
Haitao Li
Josh Blaney
Michelle Guarino
Courtney Kimoto
Mariel Young
Zhenyu Zhao
Brian Wang
Shahaf Abileah
Wayne Zhang
Jeremy Wang
Chris Zhu
Alice Zheng
Joy Zhang