Read this if you want to talk to your TV right now.

No technical knowledge needed.

Published in

Snips Blog

10 min readOct 19, 2018

A note for novices: The first section of this tutorial is all about installation and doesn’t require any technical background. The second section is mainly for developers who’d like to add advanced functionalities or make their own assistant from the ground up.

In order to talk to your TV, you’ll need a voice assistant. We‘re all concerned that the voice assistants coming from big tech companies, like the Amazon Echo, Google Home, and Siri pod, record every word we pronounce. But neither should we be left behind in home voice technology just because we care about our privacy. Which is where Snips comes into play!

Snips is challenging the idea that adding Artificial Intelligence to our everyday life means giving up Privacy.

Their voice assistant keeps your data private, on your device. And the proof? Unlike other Voice Assistants, Snips works offline!

By the end of this blog post, you’ll be able to talk to your TV, and Google won’t know if you like dogs or Jeopardy or Real Madrid :)

Part One

Bill of Materials

You’ll need the following hardware components:

Or

You can purchase the “Snips Maker Kit”. It’s easy to setup, you’ll guarantee compatibility with the components, and honestly, it’s more beautiful than just a bunch of electronic components in the middle of the room. The Snips Maker Kit doesn’t include the BroadLink IR blaster.

Broadlink RM3 Mini ~ 24$

Once you receive the material, come back and continue from here!

Step 1 — Creating your Snips Assistant

1. Create your account on the Snips console and click on Sign Up. You’ll need to download an app at some point.

2. You should be on the page below by now. Give a name to your assistant and choose English as your language (*important*).

Snips console supports (English, French, Spanish, German, Japanese, and Korean), but the application I built is in English.

You get this pop-up right after signing up.

3. Type “Snips Remote Control” on the search bar, then add an app.

Click “Add App” on the app, then click on the “Add Apps” (bottom right).

4. Now that you’re here, write “Volume up” in the right bar and press enter. You should get something like this:

We’re making sure it’s working fine.

Congrats! We’re done with our first step.

STEP 2 — Setting up your Raspberry Pi

Our second step is to set up your Raspberry Pi with Snips! If you have some technical background, just follow the Snips documentation. I’ll be doing a step-by-step detailed guide here.

1. Download the RASPBIAN STRETCH LITE ZIP file.

What’s RASPBIAN STRETCH LITE:

Raspbian OS, is an operating system for the Raspberry PI. It’s basically the “Windows” of Raspberry Pi.

Download the ZIP file on the right side.

2.Now we’re going to download an app called Etcher to “flash” in order to upload the Raspbian software onto the Raspberry Pi.

What’s Etcher:

Etcher is a graphical SD card writing tool that works on Mac OS, Linux, and Windows, and is the easiest option for most users. Etcher also supports writing images directly from the zip file, without any unzipping required. To write your image with Etcher:

Download Etcher and install it;
Connect an SD card reader with the SD card inside;
Open Etcher and select from your hard drive the Raspberry Pi .zip file you wish to write to the SD card;
Select the SD card to which you want to write your image;
Review your selections and click ‘Flash!’ to begin writing data to the SD card.

Be patient, it may take a few minutes.

Step 3 — Downloading the BroadLink App on your phone

1.Open the App, then Tap “Add device”

2.Launch “AP mode”

3.Follow the instructions

This last instruction really only means that you have to connect to the Wi-Fi, which will be provided by the BroadLink itself. Check the following screenshot:

After long pressing twice the reset button on your BroadLink device.

You’ll get this after connecting to the BroadLink device.

Choose your network, then type in your password

If you have multiple Wi-Fi networks around, keep in mind that your Raspberry Pi should be connected to the same network as your BroadLink device.

Congratulations! You’re 70% there! Much Love❤

Step 4 — Continue setting up your Raspberry Pi

Okay, It’s going to seem a bit messy, but believe in yourself.

Go to this website:

Node.js

© Node.js Foundation. All Rights Reserved. Portions of this site originally © Joyent.

nodejs.org

Download the recommended version.

Windows users need to additionally install Git. You can get it from Git’s website. If you need Help for installation watch this video.

Step 4.1 — Configure network access

The Raspberry Pi needs internet access to download the application from the Snips Platform.

Wi-Fi: After connecting your SD card to your computer:

Open the terminal.
Run these commands (copy, paste & enter)

cd /Volumes/boot
touch ssh
nano wpa_supplicant.conf

Copy and paste this and modify it according to your information:

country=COUNTRY
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
    ssid="<Your wifi network name>"
    psk="<Your wifi network password>"
}

Example: I’m in France (FR), my Wi-Fi Network’s name is: **Hi**, password: BYE.

3. To Exit: Press “control x” (to exit) then “y” (to save) then press “Enter”.

Step 4.2 — Install SAM

At this stage, you will install SAM. Sam is a CLI (Command Line Interface) that you will be using to create, manage, and deploy your assistants.

From a command shell (a.k.a. terminal) on your local machine, run the following command:

sudo npm install -g snips-sam

You’ll be asked to input your computer password first.

You are now able to run the SAM commands from your command shell.

Step 4.3 — Detecting your Raspberry Pi device on the network

Let’s try to find nearby Raspberry Pi devices on your network, using the command sam devices:

sam devicesScanning Raspberry Pi devices on the network...Found 1 device:- raspberrypi (192.168.9.2)

If SAM does not find any device, make sure the Raspberry Pi and your computer are on the same network. If you have any problems, check out Snips’ Network Setup Guide.

Step 4.4 — Connecting to your device

Connect to your device (using your device IP or hostname, and in the latter case, make sure to append .local to the hostname).

With the command: sam connect raspberrypi.local

sam connect raspberrypi.local? Enter username for the device: pi? Enter password for the device: [hidden]Connected to raspberrypi.local

This is not mandatory, but if you’re curious you can check the device status at any time with sam status:

sam statusConnected to device raspberrypi.localOS version ………………. Raspbian GNU/Linux 9 (stretch)Status ………………….. Snips Platform not installed

Step 4.5 — Installing Snips

Now that SAM is set up and connected to your Raspberry Pi, we can install the Snips Platform on it.

The Snips Platform is what runs your assistant. Listening to your voice from a microphone, it transcribes what you say into text, extracts the meaning of it, and performs an action accordingly. You can read more about this in the Snips Platform Documentation.

Start the installer using sam init:

sam initInstalling Snips Platform components. This may take a few minutes...Successfully installed Snips Platform components

This will take a few minutes, after which the device will reboot. Once ready, you can check again the status of your device. You should see the Status field updated accordingly, as well as a list of newly installed components.

Note: some service statuses may be different from the example below. Most of them will become active later in the installation process, as soon as an assistant is installed.

sam status

Connected to device raspberrypi.localOS version ................... Raspbian GNU/Linux 9 (stretch)Installed assistant .......... Not installedStatus ....................... Installed, not runningService status:snips-analytics .............. 0.55.2 (not running)snips-asr .................... 0.55.2 (not running)snips-audio-server ........... 0.55.2 (running)snips-dialogue ............... 0.55.2 (not running)snips-hotword ................ 0.55.2 (not running)snips-nlu .................... 0.55.2 (not running)snips-skill-server ........... 0.55.2 (not running)snips-tts .................... 0.55.2 (running)

Step 4.6 — Testing your microphone

Check that your microphone is working:

sam test microphone

Testing microphoneSay something in the microphone, then press Enter......

Assuming you have a speaker, if you can clearly hear what you just said, you can move on without further microphone configuration. If you can’t hear yourself, start the interactive microphone setup guide:

sam setup audio

Starting microphone setup...What microphone do you use?[1] Generic USB[2] ReSpeaker 7-Mic Array...

Once you are satisfied that your hardware setup is working, you are ready to move on to the next step, where you will deploy your Remote Controller Assistant. But first, make sure you give the right to the snips-skills:

sudo usermod -a -G dialout _snips-skills

Step 4.7 — Installing your assistant

Your assistant is now loaded with the Remote controller app you’ve selected on the console, and is ready to be deployed to your device. We will deploy the assistant with sam.

On the terminal, login to the console with your credentials:

sam login

? Enter email used on the console: username@mail.com? Enter password used on the console: [hidden]

Then, you can install an assistant created with your account:

sam install assistant


Fetching assistants done? Choose the assistant you wish to install on the device (Use arrow keys)❯ Snips Remote Control Assistant

It will then deploy the assistant to your device.

Downloading assistantFetching assistants doneNLU training doneASR training done✔ Deploying assistant to raspberrypi.local...✔ Snips assistant is now running. Say hey_snips to start!

Once you have the assistant installed, you can show the logs and test your assistant with:sam watch

Try “Hey Snips!”
To exit, press CTRL + C

Step 4.8— Teaching your Assistant

Make sure your BroadLink is on and connected: Say “Hey Snips! Please configure my BroadLink.”

Snips will search for your BroadLink device and note its information (such as its IP and Mac addresses) in order to be able to communicate with it later.

Yes, you just programmed by voice!!! Now isn’t that cool enough??

Step 4.9 — Voice Commands

Now it’s time to teach your assistant to use the buttons of your own remote controller. To do so, first say the following:

“Hey Snips!”

Our available buttons are: (AW = Also Works)

— Turning on: Say “Turn on the TV.” AW: “Light the TV”, “Put the TV”, “Switch on the TV.”

— Turning off: Say “Turn off the TV.” AW: “Sleep the TV”, “Kill the TV”, “Shut down the TV.”

— Mute button: Say “I’m trying to sleep here.” AW: “Shut it up”, “Mute it”, “Mute the TV.”

— Volume up: Say “Volume up” or “Volume up by [a number].”

— Volume down: Say “Volume down” or “Volume down by [a number].”

— Channel up: Say “Channel up”

— Channel down: Say “Channel down”

— Menu button: Say “Menu button”

— Right button: Say “Right button”

— Left button: Say “Left button”

— Enter button: Say “Enter button”

The first time you use each button, it will automatically turn on learning mode. You’ll see a yellow light on BroadLink — point your remote controller towards your BroadLink device.

After pressing the button once, he yellow light will turn off.

You’re all set!

You can start using the registered buttons!

If you press a wrong button:

Say “Learn [the name of the button]” and it will rekey the button.

Part Two— For Developers : GitHub code

In this section, I’ll add some programming details for you to be able to build it from the ground up!

If you fork the app on the console, and want to add a new functionality, read this on intents. After adding some intents, you’ll have to subscribe to them!

On python using Hermes_python:

channelup is the function to run for your action

You should change GabonV23 to your account’s username:

“ChannelUP” is the name of the command to be registered in an .ini file!

Part Three: Development ideas!

— One great improvement would be to use the numbers too! I already created an intent for it, you can improve the intent itself and make it more robust by adding some sentences.

Coding:

Snips will parse a .json file. The Slot Value you get when you want to watch the 873rd channel is 873.

At first, each numerical buttons’ signal has to be registered in .ini file in rm3_mini_controller.

Then, when you get the value, just split it into unique numbers. For example, int(‘873’) would become 8, 7, 3. Then you’ll just send each value, one after another, with BroadLink.