Controlling a sonoff light switch with the Snips voice assistant

Brad Roodt
Jun 9, 2019 · 9 min read
Image for post
Image for post

I recently discovered Snips, an amazing privacy-preserving voice assistant that runs entirely on device!! It’s also simple to set up and use.

Let’s install Snips on a raspberry pi & use it to turn on/off a light fitted with a sonoff switch.

The sonoff will be flashed with custom firmware so everything runs locally over MQTT , the same transport layer used by Snips. No cloud required, no cloud desired.

Install Snips

The Snips documentation is good with easy to follow guides at https://docs.snips.ai/getting-started but we’ll run through the install process.

The easiest way to get started with Snips is by using sam, a helper that simplifies the Snips installation. Install sam on the computer you’ll be using to connect to your pi.

Once sam is installed you can connect to your pi using your pi’s IP:

This will generate a key pair, copy the public key to key to your pi, & then ssh into the pi. Now you can install Snips on your pi using:

This will also install the mosquitto MQTT broker , which will be automatically started & run as a service alongside Snips.

Install a microphone

In order to interact with Snips you need a microphone & optionally a speaker if you want to use text to speech (TTS).

I used the ReSpeaker 2 Microphone Pi HAT & I’ve been impressed with the sensitivity of this device. It’s been able to detect voice commands from up to 4 metres away, in a quiet room.

Image for post
Image for post

Installation is a breeze.

Reboot when you’re done. After that you can setup audio for Snips with sam:

Make sure you see the Seeed soundcard in the list (rebooting should take care of this, but on one occasion I needed to restart more than once in order to see the card in the list of available devices).

Test microphone & speaker

Select the Seed card for both recording & playback and then test microphone and speaker with sam test microphoneand same test speaker respectively. I found there was slight distortion with the audio at full volume so opted to reduce the volume slightly. You can do this via alsamixer command line utility. 80% volume worked well for me.

Create a voice assistant

In order to respond to voice commands you need to create & install an assistant in the Snips console. Detailed instructions can be found here.

I found the assistant naming convention & structure a little confusing at first:

An assistant is, as you’d expect, able to respond to a variety of instructions and act on them. In Snips world, a single instruction is called an intent, which is then handled by an action. You might have multiple intents that are linked to a single task, for example controlling a light might have one intent to turn the light on, and another to turn the light off. These intents and their associated actions are grouped into what’s called an app, so the hierarchy is:

assistant > app > intents & actions

We’ll create 2 intents, one for turning the lights on, and the other to turn the light off.

While you can attach python action code to your app in the console, we’re going to handle intents in a node.js application instead.

  1. Create an assistant called ‘Home’ and click on it
Image for post
Image for post

2. Inside the home assistant, click ‘Add an app’, followed by ‘Create a new app’, naming it, ‘lightControl’.

Image for post
Image for post

3. Click on lightControl and then ‘Edit app’. Click on ‘Add intent’ and call it ‘turnLightOn’

Image for post
Image for post

4. Enter text ‘Turn on light 1’ in the textbox under training examples.

5. Create additional variations for the voice command, eg. ‘Switch light 1 on’

Image for post
Image for post

5. Click on ‘Create a new slot’ and call it ‘lightID’. Select Snips/number as the slot type, from the dropdown. A slot represents a variable in your voice command so you could issue commands like, ‘Turn on light 1’, or ‘Switch light 2 on’.

6. Select the number 1 in your intent text and choose lightID from the slot list.

Image for post
Image for post

7. Create another intent to turn off the light- it’s identical to the first one except the voice commands will pertain to switching off the device.

When you’re done, hit save. This will begin training your model! Done.

Install Assistant

Back on the command line, login into the snips console in order to install an assistant on your pi.

Login using your console email & password. Now you can install an assistant with:

This will bring up a list of assistants available for installation. Choose the Home assistant we just created.

Now, if you run sam status you should see a list of services, all of which should be running (except for possibly the analytics service).

You can test the applicaton with sam watch.

Try, ‘Hey Snips, turn on light 1’.

You should hear the wakeword detection sounds and then see some info about the intent in the console.

Troubleshooting

If you don’t hear any sound, and you have a speaker attached, make sure the soundcard is configured properly, and that you’re using the correct wake word.

If you hear the wakeword detection sounds, but your intent isn’t being handled (displayed to console), make sure the Snips skill server is running:

This was one of the few problematic areas I encountered while using Snips. Sometimes manually starting the service worked, but I did resort to an entirely fresh raspbian and Snips install on 2 occasions too.

Another issue I’ve yet to resolve, is that after rebooting the pi, the skills server doesn’t restart . The only way I’ve found to remedy this is by reinstalling the assistant with Sam.

Handling voice commands with javascript.

We’ll create a simple node.js app for handling the intent and then publish an mqtt message to turn on/off a light when an intent is inferred. This will be handled by our sonoff switch after we flash and configure it.

Update node

  1. SSH into your pi and check your arm version in the terminal

2. Check the node.js download page for the correct install associated with the arm version you have. It you have pi zero it should be ARMv6 and a pi 3B+, ARMv7.

Create a folder on your pi for your Snips action app. A great way of working with remote files is to mount the remote folder on your local machine so that you can edit and work with the files as though they were on your local machine — check out SSHFS.

Install the mqtt package

Then create a new javascript file and add the following. This connects to the mqtt broker, subscribes to relevant topics, and publishes messages for the lights in the home to respond too when our intents are observed.

The light switch

Image for post
Image for post

Sonoff switches allow us to control an appliance but they require an internet connection & connection to a cloud server. We want to keep our data local & be able to control the switch offline with mqtt, so we’ll flash the switch with the awesome tasmota firmware that enables us to do just that.

Below are a condensed set of instructions based on a fantastic SuperhouseTV video on how to flash sonoffs. Refer to the video if you’re confused about anything- you’re working with something that will be ultimately be connected to a high voltage power supply. I am not an electrical expert. Exercise caution.

  1. Make sure your sonoff is not connected to mains, and remove it from it’s housing.
  2. You need a USB TTL adapter to flash the device. The adapter also needs to have an output voltage of 3.3V. Mine has a jumper for toggling between 3.3V and 5V.
Image for post
Image for post

The EIE adapter I used has the following pins (DTR, RX, TX, VCC, CTS, & GND). The pins that we’re interested in are the RX, TX, VCC & GND.

2. Before plugging the usb into your machine, connect some female to male prototyping cables to the adapter. Flip the sonoff over to see where the male pins need to be inserted.

Image for post
Image for post

— Connect RX on adapter to TX on sonoff

— Connect TX on adapter to RX on sonoff

— Connect VCC on adapter to 3V3 on sonoff

— Connect GND to GND

3. Press and hold the power button on the sonoff and then insert the USB cable into your machine.

4. We’ll use esptool to flash the sonoff. Install it with pip (so you’ll need python)

5. Download the tasmota firmware — grab sonoff.bin file.

6. Find the port of the usb adapter:

This returned /dev/tty.usbserial-A5XK3RJT for me so this is the port I’ll use for flashing the device.

7. Navigate to the folder that you downloaded the sonoff.bin file to, and flash the sonoff with the instruction:

8. When it’s done, unplug the usb and plug it back in- the green LED will blink. The sonoff is advertising a wifi network that you can connect to, to configure the device. You should see it under your wifi network list. Select it and a configuration window will open.

9. Change the SSID and password to those of your local network, clear the AP2 password, and give the device a meaningul hostname. It’ll restart and join your network.

10. Find the ip of the device from your router so that you can connect to it.

11. Enter the ip address in your browser address bar, and you should be greeted with a page that looks like this.

Image for post
Image for post

12. We’re interested in the MQTT menu. Enter host address (this is the IP address of the pi that has the mosquitto broker installed on it, the one we installed snips on. Again, you can get this from your router’s connection list). Also enter client and topic as below.

Image for post
Image for post

13. Save and the device will restart.

If you have the mosquitto_pub client installed on your machine, you can quickly test that the device is working in the terminal, where -h XXX.XXX.X.XXX is the IP address of the pi that has the mosquitto broker installed:

The green LED should turn on and off. And that’s it. You’re all done.

The only thing left to do, is to carefully wire up your sonoff to a light and then fire up the node app we created earlier.

Snips Blog

This publication features the articles written by the Snips…

Brad Roodt

Written by

Mobile web div.loper & mountain junkie.

Snips Blog

This publication features the articles written by the Snips team, fellows, and friends. Snips started as an AI lab in 2013, and now builds Private-by-Design, decentralized, open source voice assistants.

Brad Roodt

Written by

Mobile web div.loper & mountain junkie.

Snips Blog

This publication features the articles written by the Snips team, fellows, and friends. Snips started as an AI lab in 2013, and now builds Private-by-Design, decentralized, open source voice assistants.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store