Controlling a sonoff light switch with the Snips voice assistant

I recently discovered Snips, an amazing privacy-preserving voice assistant that runs entirely on device!! It’s also simple to set up and use.

Let’s install Snips on a raspberry pi & use it to turn on/off a light fitted with a sonoff switch.

The sonoff will be flashed with custom firmware so everything runs locally over MQTT , the same transport layer used by Snips. No cloud required, no cloud desired.

Install Snips

The Snips documentation is good with easy to follow guides at https://docs.snips.ai/getting-started but we’ll run through the install process.

The easiest way to get started with Snips is by using sam, a helper that simplifies the Snips installation. Install sam on the computer you’ll be using to connect to your pi.

sudo npm install -g snips-sam

Once sam is installed you can connect to your pi using your pi’s IP:

sam connect 192.XXX.X.XXX

This will generate a key pair, copy the public key to key to your pi, & then ssh into the pi. Now you can install Snips on your pi using:

sam init

This will also install the mosquitto MQTT broker , which will be automatically started & run as a service alongside Snips.

Install a microphone

In order to interact with Snips you need a microphone & optionally a speaker if you want to use text to speech (TTS).

I used the ReSpeaker 2 Microphone Pi HAT & I’ve been impressed with the sensitivity of this device. It’s been able to detect voice commands from up to 4 metres away, in a quiet room.

Installation is a breeze.

sudo apt-get update
sudo apt-get upgrade
git clone https://github.com/respeaker/seeed-voicecard.git
cd seeed-voicecard
sudo ./install.sh

Reboot when you’re done. After that you can setup audio for Snips with sam:

sam setup audio

Make sure you see the Seeed soundcard in the list (rebooting should take care of this, but on one occasion I needed to restart more than once in order to see the card in the list of available devices).

Test microphone & speaker

Select the Seed card for both recording & playback and then test microphone and speaker with sam test microphoneand same test speaker respectively. I found there was slight distortion with the audio at full volume so opted to reduce the volume slightly. You can do this via alsamixer command line utility. 80% volume worked well for me.

Create a voice assistant

In order to respond to voice commands you need to create & install an assistant in the Snips console. Detailed instructions can be found here.

I found the assistant naming convention & structure a little confusing at first:

An assistant is, as you’d expect, able to respond to a variety of instructions and act on them. In Snips world, a single instruction is called an intent, which is then handled by an action. You might have multiple intents that are linked to a single task, for example controlling a light might have one intent to turn the light on, and another to turn the light off. These intents and their associated actions are grouped into what’s called an app, so the hierarchy is:

assistant > app > intents & actions

We’ll create 2 intents, one for turning the lights on, and the other to turn the light off.

While you can attach python action code to your app in the console, we’re going to handle intents in a node.js application instead.

1. Create an assistant called ‘Home’ and click on it

2. Inside the home assistant, click ‘Add an app’, followed by ‘Create a new app’, naming it, ‘lightControl’.

3. Click on lightControl and then ‘Edit app’. Click on ‘Add intent’ and call it ‘turnLightOn’

4. Enter text ‘Turn on light 1’ in the textbox under training examples.

5. Create additional variations for the voice command, eg. ‘Switch light 1 on’

5. Click on ‘Create a new slot’ and call it ‘lightID’. Select Snips/number as the slot type, from the dropdown. A slot represents a variable in your voice command so you could issue commands like, ‘Turn on light 1’, or ‘Switch light 2 on’.

6. Select the number 1 in your intent text and choose lightID from the slot list.

7. Create another intent to turn off the light- it’s identical to the first one except the voice commands will pertain to switching off the device.

When you’re done, hit save. This will begin training your model! Done.

Install Assistant

Back on the command line, login into the snips console in order to install an assistant on your pi.

sam login

Login using your console email & password. Now you can install an assistant with:

sam install assistant

This will bring up a list of assistants available for installation. Choose the Home assistant we just created.

Now, if you run sam status you should see a list of services, all of which should be running (except for possibly the analytics service).

You can test the applicaton with sam watch.

Try, ‘Hey Snips, turn on light 1’.

You should hear the wakeword detection sounds and then see some info about the intent in the console.

Troubleshooting

If you don’t hear any sound, and you have a speaker attached, make sure the soundcard is configured properly, and that you’re using the correct wake word.

If you hear the wakeword detection sounds, but your intent isn’t being handled (displayed to console), make sure the Snips skill server is running:

sam status

This was one of the few problematic areas I encountered while using Snips. Sometimes manually starting the service worked, but I did resort to an entirely fresh raspbian and Snips install on 2 occasions too.

Another issue I’ve yet to resolve, is that after rebooting the pi, the skills server doesn’t restart . The only way I’ve found to remedy this is by reinstalling the assistant with Sam.

Handling voice commands with javascript.

We’ll create a simple node.js app for handling the intent and then publish an mqtt message to turn on/off a light when an intent is inferred. This will be handled by our sonoff switch after we flash and configure it.

Update node

1. SSH into your pi and check your arm version in the terminal

uname -m

2. Check the node.js download page for the correct install associated with the arm version you have. It you have pi zero it should be ARMv6 and a pi 3B+, ARMv7.

wget https://nodejs.org/dist/v10.16.0/node-v10.16.0-linux-armv7l.tar.xz
tar -xvf node-v10.16.0-linux-armv7l.tar.xz
cd node-v10.16.0-linux-armv7l
sudo cp -R * /usr/local/

Create a folder on your pi for your Snips action app. A great way of working with remote files is to mount the remote folder on your local machine so that you can edit and work with the files as though they were on your local machine — check out SSHFS.

Install the mqtt package

npm install mqtt

Then create a new javascript file and add the following. This connects to the mqtt broker, subscribes to relevant topics, and publishes messages for the lights in the home to respond too when our intents are observed.

The light switch

Sonoff switches allow us to control an appliance but they require an internet connection & connection to a cloud server. We want to keep our data local & be able to control the switch offline with mqtt, so we’ll flash the switch with the awesome tasmota firmware that enables us to do just that.

Below are a condensed set of instructions based on a fantastic SuperhouseTV video on how to flash sonoffs. Refer to the video if you’re confused about anything- you’re working with something that will be ultimately be connected to a high voltage power supply. I am not an electrical expert. Exercise caution.

1. Make sure your sonoff is not connected to mains, and remove it from it’s housing.
2. You need a USB TTL adapter to flash the device. The adapter also needs to have an output voltage of 3.3V. Mine has a jumper for toggling between 3.3V and 5V.

The EIE adapter I used has the following pins (DTR, RX, TX, VCC, CTS, & GND). The pins that we’re interested in are the RX, TX, VCC & GND.

2. Before plugging the usb into your machine, connect some female to male prototyping cables to the adapter. Flip the sonoff over to see where the male pins need to be inserted.

— Connect RX on adapter to TX on sonoff

— Connect TX on adapter to RX on sonoff

— Connect VCC on adapter to 3V3 on sonoff

— Connect GND to GND

3. Press and hold the power button on the sonoff and then insert the USB cable into your machine.

4. We’ll use esptool to flash the sonoff. Install it with pip (so you’ll need python)

pip install esptool.py

5. Download the tasmota firmware — grab sonoff.bin file.

6. Find the port of the usb adapter:

ls -l /dev/tty.usb*

This returned /dev/tty.usbserial-A5XK3RJT for me so this is the port I’ll use for flashing the device.

7. Navigate to the folder that you downloaded the sonoff.bin file to, and flash the sonoff with the instruction:

esptool.py -p /dev/tty.usbserial-A5XK3RJT write_flash -fs 1MB -fm dout 0x0 sonoff.bin

8. When it’s done, unplug the usb and plug it back in- the green LED will blink. The sonoff is advertising a wifi network that you can connect to, to configure the device. You should see it under your wifi network list. Select it and a configuration window will open.

9. Change the SSID and password to those of your local network, clear the AP2 password, and give the device a meaningul hostname. It’ll restart and join your network.

10. Find the ip of the device from your router so that you can connect to it.

11. Enter the ip address in your browser address bar, and you should be greeted with a page that looks like this.

12. We’re interested in the MQTT menu. Enter host address (this is the IP address of the pi that has the mosquitto broker installed on it, the one we installed snips on. Again, you can get this from your router’s connection list). Also enter client and topic as below.

13. Save and the device will restart.

If you have the mosquitto_pub client installed on your machine, you can quickly test that the device is working in the terminal, where -h XXX.XXX.X.XXX is the IP address of the pi that has the mosquitto broker installed:

mosquitto_pub -h XXX.XXX.X.XXX -t ‘cmnd/living-room-lamp-small/POWER’ -m ‘ON’ 
mosquitto_pub -h XXX.XXX.X.XXX -t ‘cmnd/living-room-lamp-small/POWER’ -m ‘OFF’

The green LED should turn on and off. And that’s it. You’re all done.

The only thing left to do, is to carefully wire up your sonoff to a light and then fire up the node app we created earlier.