Building a Smart Mirror with Voice Recognition
Use our open-source smart mirror app to make a Smart Mirror that you can have a conversation with.
Smart mirrors are gaining popularity these days as internet-connected devices become more widely available. I’ve been interested in building one for some time. However, it was important that the mirror wasn’t just cool to look at, but genuinely useful. Of course, it had to look nice too.
I quickly realized that most of the DIY Smart Mirror projects suffered from a common problem — there was no way to interact with the mirror. I wanted to ask it questions like:
- “Will I need an umbrella today?”
- “How my stocks are performing?”
- “Turn off the bedroom lights.”
I didn’t want an app on my phone to control my mirror, because that defeats the purpose. I wanted a truly smart mirror that I could interact with.
Since a mirror is a large panel that you also have to keep clean, it was obvious that using voice recognition was the way to go.
So here’s the story of how I built a conversational smart mirror with Houndify’s Voice Recognition APIs, and how you can too.
An Overview: What are we building?
Here’s an overview of what we’re going to build in this tutorial. It’s a one-way mirror and an Android tablet enclosed within a frame. The mirror displays the current weather and time. Once you speak “Ok Hound” near it, the mirror will start listening. You can then ask queries such as:
- “Did the stock price of Apple go up?”
- “Will it rain this weekend?”
- “Who won the hockey game last night?”
The mirror will speak the responses back to you, and display them in the center of the screen. You can also say “clear the mirror” to remove the contents on the screen.
Note: Due to the angle of this shot, you aren’t able to see the screen in full-brightness but the text is legible when you are looking directly at it.
The materials needed for the project were all easy to find. I wanted to start with a prototype smart mirror that I could put up on a wall.
- A one-way mirror: You can easily find this in a glass store. I got a 16" x 20" mirror and it cost approximately $120 USD.
- An Android Tablet with a black bezel: You want the largest tablet you can find with a high DPI and a good mic running Android v5.0+. Nothing else really matters. A Google Nexus Tablet should be fine. The tablet will be the user interface of the mirror.
- Frame: Same size as the mirror that you acquired.
- Black Bristol Board
- Tape or velcro
- [Optional] Bluetooth Speaker: You may need this if your Android Tablet has a lousy speaker.
- [Optional] Wired microphone: You may need this if your Android tablet’s microphone is not sensitive enough. We used this one.
Note: If you want to build a larger version of this mirror, I recommend switching out the Tablet for a monitor + Raspberry Pi. I prefer the tablet because it comes with an SDK and the necessary hardware (mic, charger, wifi, etc.).
The software for the Houndify Smart Mirror is open-source and available on GitHub. It’s an Android App built using the Houndify Android SDK and React Native.
In this section, I’ll outline the following:
- Creating a developer account on Houndify to get a Client ID and Client Key to make voice requests.
- Setting up the Smart Mirror App on your tablet
Creating a Houndify Developer Account
The first step is to create a Houndify account and get started with the free tier. This lets you start making voice requests and receive JSON responses.
Head over to Houndify.com and sign up for an account. After signing up,go to the Dashboard, and register a new client.
Call the new client “Mirror”, and proceed to the Enable Domains page. It looks like this:
Houndify’s Voice Recognition system groups queries into programs called Domains. Enabling a domain allows your app to understand queries related to it.
For example, enabling the Weather Domain allows your client to understand and respond to weather-related voice queries such as:
- “Whats the weather today in Toronto?”
- “Will it rain this weekend?”
Go ahead and enable Weather, Sports, and Stock Market, so that the Smart Mirror app understands and responds to any voice queries related to those topics.
Then press Continue. You’ll be taken to a page with your generated Client ID and Client Key. Copy and paste those somewhere. Done.
Making my first voice request
With your Client ID and Client Key, you’re able to try out a sample voice request right in the browser. Click on the Try Houndify API page:
Click on the microphone and say “What’s the weater in Toronto?” Alternatively, just type that into the text box. You should see a real-time JSON response come back. This gives you a taste of how the smart mirror will operate. We’ll be able to ask it things, and it will be able to display and speak the responses back to us.
Building and Running the Smart Mirror Android App
In this section, I’ll discuss how to run the Smart Mirror Android app on your Android tablet. Don’t worry if you don’t know Android. It’s not necessary for this section.
First, go ahead and get the latest version of the Houndify Smart Mirror app from Github.
Next, open your Terminal and navigate to the project’s directory. Then, run the following command (you will need NodeJS to run this, so install that if you don’t have it):
This will install the React Native dependencies.
Next, install Android Studio and the Android SDK 5.0 Tools.
Once Android Studio is installed, launch it and open up the project. It’ll automatically run a Gradle Build. Once that is complete, open the Constants.java file and add your Houndify Client ID and Client Key in.
Running the App on your Tablet or Emulator
That’s all we need to do to get the basic software working, so let’s try to run it now.
For this, I recommend downloading Genymotion, which is a faster Android emulator than the stock emulator. Alternatively, you can just connect your Android tablet to your laptop and run the app directly on your device.
Once the app launches, it should look something like this:
Note: The app is black and white because when the tablet is placed behind the one-way mirror, the black portions appear as a regular reflective mirror, while the white portions appear opaque. This gives the illusion of text on a mirrored surface.
Time for action! On this screen, you can try out the same weather query that you tried earlier on the web browser. First say “Ok Hound”. That activates the voice recognition API to start listening:
Then try saying “what’s the weather in Toronto?”:
If all goes well, you should see a Weather card fade on to the screen with the response of your weather query!
Putting it together
Putting the mirror, the tablet, and the frame together is simple. The key is to make sure the tablet is centered on the mirrored surface. However, do not attach the tablet directly on to the mirror, because the glue can damage the one-way mirror film.
Here’s a diagram for how everything should fit together:
We used velcro tape to tape the tablet to the back of the frame.
Some other things you’ll probably want to do to have a nice experience:
- Make a small hole on the back of the frame to allow for the power cord and a wired microphone cord to be attached to the tablet.
- Ensure that the tablet’s display setting is set to “Always On”.
Once everything is set up, it should look like this:
You can mount the frame on a wall. We haven’t had a chance to do that yet.
Now that the smart mirror is set up and working, we’ll cover the following in the next tutorial:
- Building cards for other domains
- Adding support for custom queries
Head on over to Houndify.com to create a free developer account, and start building applications that understand natural language.