Giving Your React Native App a Voice with Houndify

Whether you’re building mobile apps for enterprise solutions, hackathons, or your latest startup, voice enabling your app is a great way to a differentiate it from other products and connect with your users in a brand new way.

Houndify’s voice technology powers many enterprise solutions including experiences from Mercedes-Benz, Hyundai, and the Pandora app (to name a few). Now, you can bring this functionality to your React Native application with the Houndify React Native SDK.

We’ll go through how to set up a simple voice app using React Native and Houndify, covering both the app itself and the server behind it.

For this tutorial, you’ll need a basic understanding of

• React Native
• NodeJS
• Express (a server library for NodeJS)

We’ll assume you’re starting your app and server from scratch, but what we do here can also be easily added to an existing setup.

Making a Houndify Account

Already have an account? Feel free to skip this section.
Otherwise, just head to Houndify and follow the instructions to get started for free.

Here’s a quick overview.

• You create a client, which is authenticated using a Client ID and Client Key.
• You enable domains, such as Weather, Music, and Car Control, that control which types of queries your client responds to.
• You can test this out online, in the “Try the Houndify API” section, or make requests using one of our many platform SDKs.

Setting up your Server

First, we’re going to set up a basic NodeJS server.

Why do I need a server? This server sits between your app and Houndify’s servers, allowing you to authenticate your app’s requests without distributing your (secret) client key in your apps.

Start by creating a new directory for your app. We’ll be using a structure like this, but you can use whatever you’d like.

my-voice-app/
 server/
 app/

For now, navigate to the /server directory, any commands we run or files we create will be here for now.

First, we need to initialize this directory.

npm init

This is a sample app, so we’ll start with the minimum dependencies required.

• Express is a great library for building servers.
• Express-WS lets Express communicate with your app in real time using WebSockets.
• Houndify lets your server communicate with Houndify’s, enabling all sorts of voice functionality.

Install these by running:

npm install --save express express-ws houndify

Great! Next, create a javascript file for your server, we recommend server.js. Copy in the following code and add in your Client ID and Client Key where indicated. You can get this information from your Houndify dashboard.

All the code you need for a simple houndify-react-native server.

1. First, we set up a basic express server and enable WebSockets.
2. Next, we import createReactNativeProxy , a function that lets Houndify communicate with React Native.
3. Finally, we tell express to send all requests to [your server url]/houndifyReactNativeProxy to Houndify.

Running the Server

You can run your server with node server.js, like any express app. If you’re running the server on a personal laptop or computer, you’ll probably need to make the server accessible to the internet since we’ll be running our app on another device. We recommend using a tool like ngrok to connect to your server remotely.

What’s ngrok? ngrok lets you connect to a local port on your computer from any other computer with internet access.

Setting Up Your App

Houndify React Native supports react-native version 0.59.8 and below. If it’s not already, install react native by running

npm i -g react-native-cli

Next, assuming your app used the same directory structure as ours, head to the /app directory we made earlier.

Once you’ve done this, create a new react-native project by running

react-native init MyProjectName — version 0.59.8

⚠️ The version number is important! You may have a later version of react-native installed by default. ⚠️

Now, if you open App.js you should see something like this.

Initial App.js

We’re making our app as simple as possible to start, so we’ll get rid of most of the App.js template. Just this should be enough.

Stripped down App.js

Adding Houndify

Great, we’re almost there! First, add houndify-react-native to your project. You can find it on npm here.

npm install --save houndify-react-native

This module contains some native code that depends on your platform (iOS or Android). To make sure this is properly configured, run the following.

react-native link react-native-audio-record
react-native link react-native-tts

In App.js add the following import near the top of the file.

import HoundifyComponent from "houndify-react-native";

HoundifyComponent takes care of almost all of the boilerplate needed to voice-enable your app. If you want some more details, check it out on npm. For now though, we’ll show you how to get up and running with it quickly.

In your render method, you can add the following.

Minimal code needed for HoundifyComponent

That’s a lot of brackets…but it’s not too bad! We’ll go through each of the variables above.

Curious about the brackets? This is a render-prop. Instead of passing children directly, we pass in a function that takes some properties. HoundifyComponent injects these properties, allowing us to pass information directly into children objects.

host
This is the URL of your server. If you’re on localhost, it’ll look like ws://localhost:3000, if you’re using ngrok it’ll look like https://{{random-id}}.ngrok.io.

transcription
While your user is speaking, Houndify will continuously send you partial transcripts to show you what we heard. This variable will always have the most up-to-date transcription.

isRecording
This is a boolean indicating whether you are currently recording.

start / abort
These are functions that you can call to start or stop recording. You may want to hook these up to a button in your app.

Put your UI code inside of this component. For a basic example, use the following.

Some simple UI code for HoundifyComponent

At this point, your App.js should look something like this.

Our super simple sample app (try saying that 10 times fast…)

We’ll improve on this in a bit, but first, let’s run it and see what happens.

Note, we added a SafeAreaView, this makes sure our app isn’t cut off by things like notches.

Running your App

We’ll be running our app in an iOS emulator. If you’d like to use Android, check React Native’s documentation to find out how.

react-native run-ios

Issues? Scroll down to the bottom of this article for some tips.

You should see an emulator pop up, and after everything is done loading, a screen that looks like this.

Our simple app at work.

Press the button and begin speaking, you should see your words transcribed on the screen, and soon, a response from Houndify.

This app is way too simple to be useful, but it was a nice clean way to introduce the library. We can keep going and make this app as sophisticated as we’d like, but no more voice-handling logic is necessary, only UI.

That’s the benefit of this library: you focus on what your user sees, we’ve taken care of the rest.

Down below, we’ll discuss some more advanced use cases.

Advanced Use

We’ll explain some of houndify-react-native’s more advanced features that give you the flexibility to build anything you can imagine. This will be a quick intro, check out npm for more thorough documentation.

More Properties

Above, we used some of the properties exposed by HoundifyComponent — transcription, writtenResponse, isRecording, and start/stop. However, you also have access to the response property, which is the full JSON response returned by Houndify.

The structure of this object depends on the type of request you make: weather, maps, etc, are all different topics that require different information. However, all responses follow this basic structure.

Some important properties from the Houndify response.

The whole response is called the HoundServer JSON, which you can learn more about here. A given request returns one or more results, known as CommandResults, with their own data. You can learn more about CommandResults here.

Your app may want to display different types of UI views for things like weather, facts, etc. You can use the CommandKind property to determine what UI to display. For example, if you ask about weather it will we "WeatherCommand", if your command isn’t understood it will be "NoResultCommand".

Also, don’t be afraid to explore for yourself and figure out what the response looks like! You can head to Houndify, select your client, and go to “Try the Houndify API” to feed in sample queries and take a look at the response.

Custom Commands

We get it. You’re building the next big thing, and you may need your app to understand queries that no other app in the world does. If it can’t already, you can easily extend Houndify to understand conversations relevant to you and your users — anything from bake me some bread for your oven to full thrust ahead for your rocketship.

Here’s a great tutorial on how to do that.

How to add Custom Commands to your Houndify Client

Event Hooks

The HoundifyComponent structure we discussed above shows a way to quickly pass information into your UI. But what if you want to do things like receive onStart/onResponse/etc events, or control the component from other parts of the app? We’ve exposed a few event handlers for you to hook on to.

We’ll just show a few handlers here, but you can see the full list on npm.

Some important event hooks on HoundifyComponent.

Volume Hook

One hook worth mentioning is the onVolume hook. This does not return a direct decibel volume, but instead a decimal representation of the voice volume. A potential use case is to use this value as a multiplier to generate UI effects.

To illustrate this use, we made a slightly more sophisticated app, also using houndify-react-native. Note, the lag in this clip isn’t the app, it’s because it’s a gif.

Using the volume hook.

This effect took only a couple of lines of code using houndify-react-native. In fact, this sample app was built by adding a couple of dozen lines of code to the one we built together.

Something not working?

• Make sure both the app and server are running, and that the host property in your app’s HoundifyComponent actually points to your server.
• Make sure you’ve added your Client ID and Client Key where mentioned.
• Check out this package’s npm for some common installation issues and how to solve them.
• Give us a shout on our community.

The Bottom Line

Adding voice to your app is a great way to add accessibility and convenience, and hey, it looks pretty damn cool too. Whether this is for a hackathon, a side project, or a startup, voice enabling can complement your existing work and give you the edge you need.

React Native lets you build fast and quickly get your idea running on the world’s most popular devices. With Houndify React Native, you can build just as quickly using the React Native tools you’re used to, while taking advantage of the same Houndify technology used by industry giants.

So why wait? Houndify your app today. We can’t wait to see what you build!

Like what you hear?

• Check out more of our Medium blog here.

Houndify

• Learn more about developing with Houndify here.

Houndify by SoundHound Inc.