Build Your First Callbot (IVR)

Introduction

Hey, I am Thomas. I am part of ING Analytics. I am working there as a machine learning / software engineer. Currently involved in our Customer Dialog Analytics Center of Expertise where we embrace the changing human-computer interaction.

In the Analytics Unit, we work on global and scalable business cases, this in order to become a digital bank. We found out that there is great value behind the use of callbots. For example, to gather information about our customer in the KYC context, to correctly route our incoming calls to the proper department. It can also be used to pro-actively call our customers to send reminders (appointments, deadlines,…).

This post tackles callbots from a technical perspective. In this regards, I would like to get you kickstarted in the world of (call)bots, by building a smart IVR.

Photo by Tim Gouw on Unsplash

Summary

Natural Language Processing (NLP), Text-to-Speech (TTS & STT), Chatbots, Voice assistants, AI Powered Customer Support… so many fancy buzzwords! Let’s experiment with them by selecting a simple use case.

Who has already called a customer service center and heard: ‘If you want to speak in Chinese press 1, if you want to block your card press 2, if you want to hear this message again press star, if you want…’

What if instead you got ‘Hello, how can I help you?’ and simply answer using your voice? Afterwards, you get directly redirected to the person (-maybe bot) that can handle your request?

This blog post aims at getting you started with the development of a simple application / proof-of-concept (POC) that can just do that.

Architecture

The architecture is a basic web application that will integrate with Twilio and Dialogflow.

The sequence diagram is as follow:

• a customer calls a number registered on Twilio side
• Twilio will use TTS to POST to our endpoints what the customer says as text
• we will pass this information to Dialogflow. Using NLP on the data we have trained Dialogflow with, we will get the matched intent as a response
• we will then pass back the response to Twilio that will play it to our customer using TTS

Set-up

Simply follow these steps if you want to create your own POC.

Dialogflow

Create your first Dialogflow agent.

Newly created agent

To define how conversations work, you create intents in your agent that map user input to responses. In each intent, you define examples of user utterances that can trigger the intent, what to extract from the utterance, and how to respond. Generally, an intent represents one dialog turn within the conversation.

Here, we have created 4 intents:

• The Default Welcome Intent that will get triggered when we received a call. This thanks to the event mechanism.
• The Default Fallback Intent, in case the bot do not understand the customer request.
• The open_account intent, in case the customer wants to open an account.
• The block_card intent, when the customer’s query is about the need to block a card.

In order to run your application later on, you need to download a service-account for Dialogflow and keep track of your project ID. Those can be found under settings.

Where to get projectNameOnGCP and serviceAccountPath

Call processing

Create an account on Twilio and set-up a phone number and webhook. The webhook is the endpoint Twilio will use to send you information about the inbound calls.

You will need to set it up as described in the image below. Based on where you host your server, you will need to change the host part of the webhook. This will be explained below.

You can find more information here.

Server

Let’s now set up our application locally using ngrok. Ngrok eases your development by creating a tunnel from a secured public URL to your localhost. It allows the application on your machine to be reached by Twilio for testing. Make also sure that you have nodeJS and Git installed.

Afterwards, you can run the following command:

Change your hostname in Twilio with the one you have received from ngrok.

Let’s now perform a simple test by calling your Twilio number. If everything goes smoothly, you should get the answer you have inserted in Dialogflow.

You now have a working POC! Later on, you might want to host your application on Heroku, AWS or Google Cloud to ensure some availability 😊

Note that this application is not production-ready! There is no proper error-catching mechanism, the endpoints are unsecured, we are missing (security) events monitoring,…

What’s next?

Hopefully you succeeded the steps above or at least have some understanding of what is feasible using NLP, TSS and STT. For us, this application has been the start of a really exciting journey in the world of callbots.

Finally, with some engineering efforts, you can enhance your application so that it becomes a real platform also supporting Google Home, Alexa, SMS-bot, Chat-bot, etcetera!

Feel free to reach out if you have any questions!

Thomas

Contact me!

Remark: Dialogflow and Twilio have been selected in this case. However, they both have lots of alternatives that are worth having a look at. There are also great open-source initiatives such as Rasa for the NLP part and Asterisk for the phone gateway part!