Building a Smart Mirror with Voice Recognition

Use our open-source smart mirror app to make a Smart Mirror that you can have a conversation with.

Constructing the Smart Mirror

Smart mirrors are gaining popularity these days as internet-connected devices become more widely available. I’ve been interested in building one for some time. However, it was important that the mirror wasn’t just cool to look at, but genuinely useful. Of course, it had to look nice too.

I quickly realized that most of the DIY Smart Mirror projects suffered from a common problem — there was no way to interact with the mirror. I wanted to ask it questions like:

  • “Will I need an umbrella today?”
  • “How my stocks are performing?”
  • “Turn off the bedroom lights.”

I didn’t want an app on my phone to control my mirror, because that defeats the purpose. I wanted a truly smart mirror that I could interact with.

Since a mirror is a large panel that you also have to keep clean, it was obvious that using voice recognition was the way to go.

So here’s the story of how I built a conversational smart mirror with Houndify’s Voice Recognition APIs, and how you can too.


An Overview: What are we building?

Here’s an overview of what we’re going to build in this tutorial. It’s a one-way mirror and an Android tablet enclosed within a frame. The mirror displays the current weather and time. Once you speak “Ok Hound” near it, the mirror will start listening. You can then ask queries such as:

  • “Did the stock price of Apple go up?”
  • “Will it rain this weekend?”
  • “Who won the hockey game last night?”

The mirror will speak the responses back to you, and display them in the center of the screen. You can also say “clear the mirror” to remove the contents on the screen.

Note: Due to the angle of this shot, you aren’t able to see the screen in full-brightness but the text is legible when you are looking directly at it.

Gathering Materials

The materials needed for the project were all easy to find. I wanted to start with a prototype smart mirror that I could put up on a wall.

  • A one-way mirror: You can easily find this in a glass store. I got a 16" x 20" mirror and it cost approximately $120 USD.
  • An Android Tablet with a black bezel: You want the largest tablet you can find with a high DPI and a good mic running Android v5.0+. Nothing else really matters. A Google Nexus Tablet should be fine. The tablet will be the user interface of the mirror.
  • Frame: Same size as the mirror that you acquired.
  • Black Bristol Board
  • Tape or velcro
  • [Optional] Bluetooth Speaker: You may need this if your Android Tablet has a lousy speaker.
  • [Optional] Wired microphone: You may need this if your Android tablet’s microphone is not sensitive enough. We used this one.
Note: If you want to build a larger version of this mirror, I recommend switching out the Tablet for a monitor + Raspberry Pi. I prefer the tablet because it comes with an SDK and the necessary hardware (mic, charger, wifi, etc.).

The Software

The software for the Houndify Smart Mirror is open-source and available on GitHub. It’s an Android App built using the Houndify Android SDK and React Native.

In this section, I’ll outline the following:

  1. Creating a developer account on Houndify to get a Client ID and Client Key to make voice requests.
  2. Setting up the Smart Mirror App on your tablet

Creating a Houndify Developer Account

The first step is to create a Houndify account and get started with the free tier. This lets you start making voice requests and receive JSON responses.

Head over to Houndify.com and sign up for an account. After signing up,go to the Dashboard, and register a new client.

My Houndify Dashboard. You can see the “Mirror” client here.

Call the new client “Mirror”, and proceed to the Enable Domains page. It looks like this:

Houndify’s Domain Selection Page. I have “Weather”, “Sports”, and “Stock Market” enabled.

Houndify’s Voice Recognition system groups queries into programs called Domains. Enabling a domain allows your app to understand queries related to it.

For example, enabling the Weather Domain allows your client to understand and respond to weather-related voice queries such as:

  • “Whats the weather today in Toronto?”
  • “Will it rain this weekend?”

Go ahead and enable Weather, Sports, and Stock Market, so that the Smart Mirror app understands and responds to any voice queries related to those topics.

Then press Continue. You’ll be taken to a page with your generated Client ID and Client Key. Copy and paste those somewhere. Done.

Making my first voice request

With your Client ID and Client Key, you’re able to try out a sample voice request right in the browser. Click on the Try Houndify API page:

Trying out the Houndify API by speaking into my laptop, and receiving a JSON response.

Click on the microphone and say “What’s the weater in Toronto?” Alternatively, just type that into the text box. You should see a real-time JSON response come back. This gives you a taste of how the smart mirror will operate. We’ll be able to ask it things, and it will be able to display and speak the responses back to us.

Building and Running the Smart Mirror Android App

In this section, I’ll discuss how to run the Smart Mirror Android app on your Android tablet. Don’t worry if you don’t know Android. It’s not necessary for this section.

First, go ahead and get the latest version of the Houndify Smart Mirror app from Github.

Next, open your Terminal and navigate to the project’s directory. Then, run the following command (you will need NodeJS to run this, so install that if you don’t have it):

npm install

This will install the React Native dependencies.

Next, install Android Studio and the Android SDK 5.0 Tools.

Once Android Studio is installed, launch it and open up the project. It’ll automatically run a Gradle Build. Once that is complete, open the Constants.java file and add your Houndify Client ID and Client Key in.

Replace the Client ID and Client Key in Constants.java with your Houndify.com credentials.

Running the App on your Tablet or Emulator

That’s all we need to do to get the basic software working, so let’s try to run it now.

For this, I recommend downloading Genymotion, which is a faster Android emulator than the stock emulator. Alternatively, you can just connect your Android tablet to your laptop and run the app directly on your device.

I set up a Genymotion emulator with these specifications.

Once the app launches, it should look something like this:

Smart Mirror App running on the Genymotion Emulator.
Note: The app is black and white because when the tablet is placed behind the one-way mirror, the black portions appear as a regular reflective mirror, while the white portions appear opaque. This gives the illusion of text on a mirrored surface.

Time for action! On this screen, you can try out the same weather query that you tried earlier on the web browser. First say “Ok Hound”. That activates the voice recognition API to start listening:

Once OK Hound is spoken, the app starts listening for a query.

Then try saying “what’s the weather in Toronto?”:

The Smart Mirror providing a response to my weather query.

If all goes well, you should see a Weather card fade on to the screen with the response of your weather query!

Putting it together

Putting the mirror, the tablet, and the frame together is simple. The key is to make sure the tablet is centered on the mirrored surface. However, do not attach the tablet directly on to the mirror, because the glue can damage the one-way mirror film.

Here’s a diagram for how everything should fit together:

Diagram of how the components fit together

We used velcro tape to tape the tablet to the back of the frame.

Some other things you’ll probably want to do to have a nice experience:

  • Make a small hole on the back of the frame to allow for the power cord and a wired microphone cord to be attached to the tablet.
  • Ensure that the tablet’s display setting is set to “Always On”.
Setting up the tablet with the back of the frame.

Once everything is set up, it should look like this:

You can mount the frame on a wall. We haven’t had a chance to do that yet.

Part 2

Now that the smart mirror is set up and working, we’ll cover the following in the next tutorial:

  1. Building cards for other domains
  2. Adding support for custom queries

Head on over to Houndify.com to create a free developer account, and start building applications that understand natural language.