Infrared Voice Controlled With Snips
A guest blog post by Greg.
Too many remotes? Simplify! Use your voice to control infrared devices around your home using Snips voice platform. A completely on-device assistant that does not share anything it hears, or what it is you do. Privacy by design.
Background
My young 3-year-old likes to hide my TV remote and I can never find it when I need to use it. I already use Snips voice platform in my home, so I started thinking about how I could remove the remotes completely and just ask my home assistant, with my voice, to control my devices. Turns out it was very easy, and this project is so simple, easy to setup, and manage, that you can too.
This project not only benefits me, it also fills a requirement for other users out there. My elderly parents, for example, can never find the right button to press on the TV remote. This also benefits people with disabilities who are not able to manipulate a remote physically, but are able to say to an assistant what they want to do.
Once upon a time… I had 4 remotes, just for the devices in the lounge room alone. A remote each for the TV, amplifier, DVD, and aircon. They are now in the bottom draw, and I just speak aloud what I want to control.
Some basics
This project uses Snips. So a very quick run down of “what is Snips?”. Snips is natural language understanding and speech recognition software. Your interaction with Snips as a developer is as a set of applications that together create an AI assistant and is able to run on a tiny Raspberry Pi 3B or a range of other devices. Snips listens for the wake word to be spoken for the assistant to know you want it to listen to you. It then converts what you say into text (TTS and ASR), then parses the text into meaningful intents (NLU). Applications, such as this project, listen for a specific intent notification from Snips. When they receive an intent, they go to work and “do something”, whatever it is they are programmed to do.
“Hey Snips, turn my Samsung TV on”
’hey snips’ = wake word — start listening to my command
’turn my Samsung TV on’ = Intent — IR Control
Specific things about the intent that are important. (1) ‘Samsung TV’ and (2)‘on’. The IR Control application (this project) sees that you want to use the Samsung TV remote and press the power ON button, so that’s what it does.
There are two parts to this project. The IR Control and the IR Client.
The IR Control is part of a Snips assistant application.
The IR Client uses a RPi Zero with an IR receiver and emitter connected to GPIO pins. This project shows you how build and interface the RPi with an IR receiver and emitter, configure the remotes you use in your home, and then apply this information to your own Snips assistant.
If you don’t yet have a Snips setup, Snips with Seeed provide a Maker Kit to get you up and running with Snips.
For more information and an in-depth look into Snips, check out the Snips Docs.
Building the IR Client
The IR Client uses a small, inexpensive Raspberry Pi Zero W (to use wifi to communicate with the IR Control in the Snips assistant). You can however use any RPi you wish.
Items you will require to build the IR Client:
- Raspberry Pi Zero (W version for wifi connection)
- Infrared Receiver Module — Generic 38kHz
- Breadboard (generic)
- 2x 940nm Infrared Emitters
- General Purpose Transistor NPN (PN2222A will do)
- 10k ohm Resistor
- Jumper wires (3x female-female, 3x female- male, 2x male-male)
Lay them out on a breadboard, as shown in the below images and circuit diagrams.
Follow the layout, paying attention to the Tx and Rx signal pins. 3.3v is used by the IR receiver, and the IR emitters use 5v.
Installing the IR Client software
The IR Client runs a small python application, which auto starts at boot. It listens for instructions from the IR Control in the assistant, and then transmits IR codes via the infrared emitters.
To install the IR Client on the RPi Zero, ssh into the device and run these commands:
git clone https://oziee@bitbucket.org/oziee/irclient.git
cd irclient
sudo bash install.shThe install.sh script:
- creates a venv environment
- installs the python library requirements
- installs LIRC
- sets up the IR Tx/Rx pin assignment
- creates the service to start the app on boot
Once the install completes, it will prompt you to edit the settings.ini file in the current directory.
nano settings.iniEnter in your Snips connected MQTT broker address and port, and a unique name for the IR Client device.
Testing the LIRC
You can now test the LIRC install on the IR Client RPi. Use the commands below and a remote control.
sudo /etc/init.d/lircd stop
mode2 --driver default --device /dev/lirc0Press some buttons on the remote control while it is pointed at the receiver and you should see the RPi console outputting multiple lines like the below:
Adding remote codes to LIRC
Adding your own remote controls yourself is simple.
sudo /etc/init.d/lircd stop
sudo irrecord -d /dev/lirc0 ~/lircd.confFollow the on screen instructions. The gist of it is, you enter in a name for the remote you are recording, press random buttons on the remote over and over a few times while LIRC learns and detects pulses, signals, pause timing etc.
You will be prompted to enter in names of the function button to record such as KEY_POWER, KEY_VOLUMEDOWN. LIRC uses capitals and a list of valid values can be found here.
Once you have completed recording your remote, you need to copy the created file to the lirc directory which contains all the conf files. You can do a `ls` to check for the name of the file that was created. It will be something like <what you named the remote>.lirc.conf
sudo cp ~/<name of remote>.lirc.conf /etc/lirc/lircd.conf.d
sudo /etc/init.d/lircd startCreate as many remote conf as you like, making sure they are all uniquely named, and are all copied over to the “/etc/lirc/lircd.conf.d” directory.
LIRC has a repo of remote configurations created by others that can be downloaded and used. Of course, not every remote on earth has been documented and available for download. Check out the repo for a list of remotes that can be downloaded for your use.
LIRC also includes command line tools to search this repo and install them, you can read how to here.
Setting up the IR Control in the Snips assistant
In your Snips assistant console https://console.snips.ai, click “add an app”.
Use the search field to look for “ir control” and then add this app to your assistant:
Once the app is installed, and the Snips skill server has taken control of running the Snips apps, you can log into the IR Control web management page from your browser by going to <url of snips server>:7777 (the default port is 7777, but you can change this to anything from inside the Snips config.ini for the app.
The management page will communicate with all the IR Clients you have built on the network, and gather a list of all the known remotes for each device and their function button mappings.
Here you are able to add common word and names to remotes and buttons which you say out loud to your assistant. Snips will then do its magic to understand what you ask and, in turn, inform the IR Controller of which device, or remote and which button IR code you want transmitted.
In the web page enter in synonyms, or words that describe or name something. In the image example above, it would be hard to ask your assistant in a sentence that you want to use the N2QAYB000352 remote, so entering in a common word or name that you would say in conversation with the assistant makes sense. You can then say “TV” or “Samsung TV” and Snips and the IR Control app will know you mean the N2QAYB000352 remote. Same goes for the function buttons. Saying “on” or “off it knows to use the KEY_POWER button that is mapped.
“Turn the TV on” = N2QAYB000352 | KEY_POWER
Creating Your Own Actions
Actions are a group of remote commands to run in series, but instead of saying each item to preform one at a time to the assistant with your voice, you can say one command to the assistant and all the grouped function buttons of each remote in the action will be performed in sequence.
Asking your assistant, “let’s watch a movie”, the TV will turn on, the amplifier turns on. Yet the amp takes 8 seconds to start before it will react to any other remote commands, so a pause value is set, as in the example image above. A 10-second pause occurs before the next function is performed, which is to set the amp to the TV input.
Check the bitbucket repo README for extra information on the web interface.
Telling Snips assistant your setup
Over in the Intent Builder (selected in the menu) all the properties for all the IR Client devices, each remote, each button, and all the synonyms are displayed in such a way so you can easily copy them over to the Snips Console assistant to fill the slot values for the training data.
One at a time, you’ll copy the items between the *****’s and import it into the corresponding Intents slots in the Snips Console. In the above image the highlighted items are copied and imported to the `ir_control` intent, into the slot button (shown below).
Once all the slots for ir_control and ir_action have been imported, save your assistant, and install the newly updated assistant to your Snips setup.
In future, if you rename a remote, delete a remote, add or remove a remotes function buttons, created new actions — you will need to use the Intent Builder to gather the changes to import into the Snips assistant. You should remove all the current values in the Snips console intent slots before importing new items, as this will remove old items that may no longer be valid.
Other Features
IR Control is also able to start a sequence of button events. If you find the need to open the TV guide on the TV and you want to move around the listings of channels and shows if would be annoying to continuously have to repeat the wake word and say a command over and over. The sequence event solves this as you can just announce function after function.
“Hey Snips, start a sequence for me” — listening for your commands.
“press the TV guide button”
“press up button”
”up”
”up”
”select”
“ok stop listening for commands” — this ends listening. If you do not let the assistant know you want to stop announcing commands, it will just keep listening until you do.
You need to say either the device or the remote once to set a base of where to start. From then on you can just say the button commands to be completed which are mapped to that device or remote. If you want to then start interacting with a different remote, say that remotes name so that IR Control knows you now will be referring to a different remote to operate. When you want to stop using commands, just ask it to stop.
In Action
Quick demo in action. The video was made in a darkened room to demonstrate the IR emitters lightly light up as they transmit.
What next
Design and print a case to hold the RPi Zero and provide IR transmitter receiver mounts.
Start thinking about what you too could automate. The possibilities are endless, limited only by to your imagination. If you can think it, code it, make it, then use it with your own home assistant. Check out the Snips store in the console for many other applications designed by other Snips developers for you to use.
Credits
Thanks to Snips (https://snips.ai) | LIRC team (http://www.lirc.org) | icons8 (https://icons8.com)
