Building a phone assistant with Voximplant and Dialogflow

I have already written a post about a phone bot that can be built using Voximplant and API.ai, but, since Google Cloud has been recently upgraded and Dialogflow has been introduced, it looks like a good idea to write about the integration with Dialogflow as well.

This post explains the principles of creating a phone assistant / bot using the platforms, so the dialog will be rather simple:

Assistant: Hi, my dear friend! Ask me about the weather in San Francisco or other city.

Caller: What is the weather in San Jose?

Assistant: It’s sunny and warm in San Jose today.

Creating Dialogflow agent

Dialogflow is created by Google, so you’ll need a Google account to use it. Visit https://dialogflow.com and register there. Everything in Dialogflow works within an “Agent” (aka Bot). There are no agents right after the registration, but there are prebuilt agents which can be used only after you create your first agent. Create new agent and call it “TestAgent”

No need to change anything else — we can proceed with the default values. Google project means your Google cloud project (a new one will be created if you don’t have any). Click Create. In the vertical menu on the left there is Prebuilt Agents option, find Weather agent and click Import. Now we have the Weather agent!

It’s easy to try it in action in the right panel, just enter or say “What is the weather in San Francisco?”

COPY CURL link will copy a command to your clipboardс:

curl ‘https://api.dialogflow.com/v1/query?v=20170712&query=What%20is%20the%20weather%20in%20San%20Francisco%3F&lang=en&sessionId=93882e69-fdeb-417d-9f73-7347c8e57a2f&timezone=Europe/Moscow' -H ‘Authorization:Bearer 1799227ce77f446fa8d64749f70527ca’

Please notice, that the text we are sending should be URIencoded and the Bearer should be sent in headers. If a context was specified for the request it’s a good idea to send a unique session id in every request to avoid context mixing.

The answer on the request looks as follows:

Voximplant’s VoxEngine code that makes this request:

Getting the info about the weather

Now when we know the city we can get the weather info. One of the options is to use API from openweathermap.org. Register and get your API-key to be able to make the requests:

curl 'https://api.openweathermap.org/data/2.5/weather?q={CITY NAME}&lang=en&appid={API_KEY}'

We will get a very simple description of the weather from the answer using the description field:

Now we have the info that can be used to generate speech with Text-to-Speech and sent into the call using Voximplant.

All together now..!

It’s time to finish our Javascript (VoxEngine) scenario. So, what this whole thing is about?

1. User calls a phone number (this number is bought in Voximplant).
2. The scenario synthesize the following prompt “Hi, my dear friend! Ask me about the weather in San Francisco or other city.”
3. The user asks, “what is the weather in {CITY NAME}”, Voximplant converts voice into text and then we send the text to Dialogflow.
4. Checking if we have a city name in the response . If something went wrong — asking again.
5. If there is a city in the response from Dialogflow — making HTTP-request to openweathermap.org to get the weather info.
6. As soon as we got the repsonse synthesizing the phrase “It’s {weather description} in {CITY NAME} today” and then hang up the call.

The following scenario code will be doing everything described above:

The last thing required is to create Voximplant application in Voximplant control panel and then connect the scenario and a phone number to it (see the details here). You can also see it in action by calling 888-7053282 (US number).