Integrating Houndify and Twilio

Using Houndify and Twilio to respond to text messages and phone calls.

It is common to integrate Houndify’s conversational intelligence and Voice AI with web, mobile, and IoT applications.

However, I recently came across a great solution to integrate Twilio with Houndify to allows users to make voice and text queries over their cellular network. This can be useful for situations where you want to search for the nearest restaurant, get flight status or even call an Uber while you are traveling or don’t have access to stable (or any) internet connection.

Here is how you can build your own smart application using Twilio and Houndify.

Step 1: Create a Houndify Account

Once the account is created, choose your default client and under the domains section, enable all relevant domains that you may want to query against.

If you’re unsure how to go about creating an account and enabling domains, check out this tutorial:

Video Walkthrough: Creating your First Client

Enable all domains that you want to ask questions for on the Houndify Interface.

Step 2: Create a Twilio Account

Head over to twilio.com to create an account. You will get a free phone number and $10 in Twilio credit to get you started.

Step 3: Create a new Phone number via the ‘Manage Number’ link.

Ensure the number that you pick has voice and text capabilities enabled.

Step 4: Clone the Call-Houndify GitHub repository

This Github repository uses the Houndify JavaScript SDK and the Twilio API to facilitate communication between the two services.

soundhound/Call-Houndify

To use this repository, you must have NodeJS and npm installed on your computer.

Step 5: Update your Client ID and Client Key

Copy and past your Houndify Client ID and Client Key from your Client’s Overview and API Keys page into config.json.

{    
"clientId": "YOUR_CLIENT_ID",  
"clientKey": "YOUR_CLIENT_KEY",    
"port": 3000,   
"https": false
}

Step 6: Configure an API endpoint for TwiML webhook for sending texts

At this point if you run your application on http://localhost:3000, it will initialize the Houndify client and interact with Houndify Backend but it will not be connected to your Twilio number. Let’s set that up.

You will have to configure a Webhook for Twilio to post the text responses to. Twilio needs to access this webhook, so it can’t exist on your localhost. At this point you can deploy your application to Heroku, Azure or AWS.

If you want continue testing locally, you can use ngrok — a secure introspectable tunnel to localhost webhook development / debugging tool. Another option is Forward. Both of these tools will publish a temporary URL exposing your localhost to the internet.

Let’s assume we ran ngrok and receieved the temporary URL: http://4873745c.ngrok.io , which points to our local web server. The Call-Houndify repository will expose a route, /sms, that should be used as the webhook.

Once you have the endpoint ready, configure it for your active Twilio phone number as shown in the screenshot below.

Step 7: Test a full loop with a text message

At this point, you should have the capability to:

1. Send a text message to the phone number that you set up
2. Receive the query to your running node instance
3. Trigger Houndify to perform a text search on the text message
4. Get a JSON response from the Houndify Backend, if the query matches one of the domains
5. Once this response is received, TwiML should send it back to the user’s phone where the text came from

Try sending a question and you should get the response back.

Step 8: Setting up Voice Queries

Now that we have text messages working, let’s try to get voice queries working as well. This part is a little more involved. There are two routes in app.js that handle voice requests: /voice, and /userquestion.

The /voice route is triggered from the ‘Voice and Fax’ webhook that initializes another instance of TwiML to greets the user. Once the user creates a voice query (asks a question), TwiML records that question and makes the recording available over a hosted URL. This is when the second route /userquestion is triggered and the reader starts streaming that question to Houndify voice search method.

const twiml = new VoiceResponse();
twiml.say(
  {voice: 'alice'}, 
  `Welcome to Call the Web powered by Houndify. How can I help you   today?`
);
twiml.record({
  action: ' [ADD YOUR NGROK OR CLOUD SERVICE PROVIDER URL HERE]/userquestion',
  timeout: 1,
});
response.type('text/xml');
response.send(twiml.toString());

Upon receiving the response, the /userquestion route is triggered again to receive the next question from the user and this process continues to loop until user disconnects.

Setting up the webhook to support voice queries on Twilio.

That’s all there is to it. Once you are ready, you can deploy the app to a cloud service and have it receive questions from everyone who calls / texts your Twilio number.

If you have any questions, feel free to leave a comment or an issue on the GitHub repository.

Happy Coding!