How to Use Your Alexa to Water Your Plants or Even Feed Your Cat
--
By day I’m a general partner at Initialized Capital helping founders and investing in startups. But I’ve never given up a hunger to build things and hack on projects in my spare time.
One of my interests at the moment is in voice-centric interfaces (as I talk about in the interview above with Harry Stebbings of 20 Minute VC).
So I’ve been spending time experimenting with Alexa’s platform. In that spirit, I wanted to share a quick tutorial:
Have you ever wanted to water your plants, open your window shades, or feed your cat simply by speaking the instructions?
In this tutorial, we’ll walk through the basics of how to set up an Alexa Skill to control a Raspberry Pi or Arduino to do all of this and more. The tutorial is intended for engineers and should take about an hour.
Both the Arduino and Raspberry Pi have general-purpose input/output pins, or GPIO for short. GPIO pins are used to connect with other physical devices, such as lights or motors. These can be toggled between ‘On’ (high) or ‘Off’ (low states). In this tutorial, we’ll connect the GPIO pin to a relay with an LED light that will helps us verify that our code is working. You can use the GPIO for binary toggling or expand the code to any other functionality afforded by Arduino and Raspberry Pi (playing music, watering plants, robot army, etc.).
To send commands to our Arduino or Raspberry Pi, we’ll combine an Alexa Skill built with AWS Lambda with PubNub. Here’s an outline of how it works:
- Alexa listens for a voice command
- Alexa pings a AWS Lambda end point with the intent/command
- Lambda publishes the command to a dedicated PubNub channel
- The Arduino/Raspberry Pi is subscribed to the PubNub channel and is “listening” for any new commands
- The Raspberry Pi or Arduino receives command from PubNub and toggles GPIO Pin
0) Set up
In order to have a communication bridge between the Amazon Lambda function and the Raspberry Pi/Arduino we’ll need a shared communication channel that both can publish and subscribe to. Pubnub provides this service for free and makes it easy to use.
You’ll need to create a PubNub account and a new PubNub application. Let’s call application “GPIO Control”. Click into GPIO “Control” and you’ll see a “Demokey” set. Make note of the “Publish Key” and “Subscribe Key” we’ll need it shortly.
1) Set Up An Amazon Alexa Skill
This is a detailed overview of setting up an Amazon Skill. If you’ve done this before, just copy the intents, and sample utterances, and skip to the lambda section. Remember to record your app id and keep it handy.
High level what we’re doing here is telling Alexa what phrase (“invocation”) we want to activate our app (“skill”) and then what commands (“intents”) to listen for. We also specify what end point it should ping when it hears an intent, in our case Lambda.
Make a new app
Go to https://developer.amazon.com/edw/home.html#/ click on “Alexa” on the top navigation, then select “Alexa Skills Kit” and on the “Add a New Skill”. Now we’ll give Amazon some information about the app, don’t worry about this as you can change all of it later:
- Leave skill type as “Custom Interaction Model”
- Type in “GPIO Control” for the name
- Type in “Control” for the invocation (this is what we’ll say to activate this app)
- Leave the other fields at “no” since we’re not using any of that
Click “save” and then the application ID will be generated. Keep this handy. We will need it for the Lambda function.
Add Intents
On the next page you should see Intents. This is the actions your skill can take. In our case we want to be able to turn off and on particular GPIOs
For intents put in:
There’s a new schema builder available now, but to keep things easy let’s just use simple JSON. Now let’s enumerate what can be given to our “turnOn” and “turnOff” commands:
Under “Enter Type” put in:
LIST_OF_GPIOS
In the “Enter Values” textbox put in our individual GPIO pins (you can expand this to however many you have, I’m only using #5 in this example):
2
3
4
5
Click “add”.
This lists the types of inputs (in our case pin numbers) you can give to your “TurnOn” / “TurnOff” command. You can add more GPIO pins if you have more. For example, if you were selecting a type of tea this could be the list of teas or if you’re controlling.
Define Sample Utterances
Now you’ll need to define what you say in order to turn the GPIO pins on or off. In the textbox under “Sample Utterances” put in:
TurnOn Turn on {GPIO}
TurnOn Turn {GPIO} on
TurnOff Turn off {GPIO}
TurnOff Turn {GPIO} off
AllOff Turn everything off
AllOn Turn everything on
AllOn All on
Feel free to add more. The format here is:
*your intent* (must be one of those you added above) *how to invoke it*
In our example “All On” and “Turn everything on” both invoke the same intent (AllOn). To turn on pin 5 — you would say “Turn on five” and it would invoke “TurnOn” command with the GPIO value 5
Here’s an what my screen looked like:
When you are done, click next to go to the next screen
Define the endpoint
This is where you’ll tell Alexa where to ping when it gets one of those commands. It can be any endpoint. To make things easy we’ll use a Lambda function so we only pay when it’s activated as opposed to keeping it running all the time.
Select AWS Lambda ARN, “North American” for region and then we’ll detour to setting up the Lambda function. We’ll come back to enter in the AWS Lambda ARN. Remember to make note of your Alexa app id as we’ll need it in the Lambda code.
2) Setting up Lambda
AWS Lambda acts as a serverless API. It lets you run code without provisioning servers ahead of time — think of it as an on-demand end point. This is perfect for what we’re doing since we don’t need a full-time server, just something to relay messages from amazon echo to pubnub.
Configuring Lambda
To set it up go to https://console.aws.amazon.com/lambda
Click “Create a Lambda function.” You’ll see a screen asking you to select a blueprint.
Click on “Blank Function.”
Next you’ll be asked to configure the trigger. We want to trigger it from Alexa passing on a voice command and execute code. So for trigger we’ll choose “Alexa Skills Kit”
Now we’ll name our function (eg. “controlgpio”) and select “Node.js 6.10” for the runtime.
Last step is to configure “Lambda function handler and role”, keep it on the defaults:
You should be all set to create your function. Hit “next” and “Create Function.”
At this point you should have an ARN generated that we can use as the “endpoint” for our Alexa Skill. This is where Alexa will ping when the Alexa skill is turned on or hears one of our intent.
To do this go back to Amazon Alexa page (https://developer.amazon.com) and paste the ARN that was just created into the textbox asking for an end point:
Hit next and you should be all set!
Lambda Code
Go back to Lambda page and provide the code for handling the commands. I’ve made it into a zip to make it easy.
The zip contains:
- Our code for getting a request from Alexa and passing it along to PubNub;
- The base Alexa library; and
- Required node libraries
After you download the .zip, go to index.js and modify the app id to the Alexa app id (see above) and PubNub keys (from the first step). You won’t need to change the other files.
Looking at the code you can see that it handles incoming requests (from Alexa), puts them into json and publishes them to a PubNub channel. We will then subscribe to that PubNub channel and listen for any requests. It will be the same for Raspberry Pi or Arduino since it’s just communicating with the PubNub channel.
I’m handling 4 cases:
- Turning on a particular GPIO (1)
- Turning off a particular GPIO (2)
- Turning all on (3)
- Turning all off (4)
The intents are being translated from words to ints to make it easier for the arduino to handle (mine has very limited memory).
To try out other intents just add them to the GPIO.prototype.intentHandlers array, rezip and reupload. You can see lots of examples in the official Amazon documentation: https://github.com/amzn/alexa-skills-kit-js
When you’re done modifying the app id and PubNub keys, save it, zip all the files — node_modules, index.js and alexa.js (don’t include the containing folder) and then upload the zipped file to Lambda.
You can read more here if you’re interested — https://docs.aws.amazon.com/lambda/latest/dg/nodejs-create-deployment-pkg.html
Testing
Now let’s make sure everything is working!
First we’ll verify that there’s nothing horrendously wrong with the lambda code and we can start a new session. To test this select action -> “configure test event”
Paste this into the code section over the existing code (it’s a modified version of the start session template)
Change the applicationId to the Alexa skill you created and hit “save and test”. It should have a green box saying “Execution results: succeeded”.
Now let’s simulate a request from to turn on GPIO 5 (Once again you’ll need to modify the app id to be your app id):
eg.
You should see the success message and if you expand the logs it should say:
{
“version”: “1.0”,
“response”: {
“outputSpeech”: {
“type”: “PlainText”,
“text”: “Turning on 5”
},
“shouldEndSession”: true
},
“sessionAttributes”: {}
}
Yay! We should also make sure it was successfully published to the PubNub channel:
Log back into your PubNub account, select your app and then on the left hand side (at the bottom) click on “debug”. You might already have a client subscribed “hello_world” otherwise just add it and verify that it’s successful by looking for the message being passed to the channel:
You should see your request (1 is the action to “turn on” a particular gpio) published to the channel you’re subscribed to. If you don’t see it make sure you have the right pub/sub keys and you’re communicating with the same channel.
Now let’s test all the pieces and make sure the whole flow works by simulating a request from Alexa. In the “service simulator” on the “test” tab in your Amazon Skill pass in the following command:
Turn on 5
To simulate GPIO 5 being switched on:
You should see a response from Lambda and a command in your PubNub debug interface.
Success! Now that we have publishing working lets subscribe to it on the Raspberry Pi and Arduino.
3a) Arduino
Connect to the Internet
First we need to connect Arduino board to the Internet over WiFi. I used an ESP8266 WiFi shield. It doesn’t come with headers soldered to the board, so just buy them separately and solder them on. Then attach the shield to your Arduino. You could also use an ESP8266 module and wire it up directly.
Then we’ll use the SparkFunESP8266WiFi library to connect your shield to the internet.
To include the library via the Library Manager, from the drop down select Sketch -> Include Library -> Library Manager and then search for “SparkFunESP8266WiFi” and restart the Arduino IDE. A more thorough guide can be found here: https://www.Arduino.cc/en/Guide/Libraries
Test that your board connects to WiFI with the following code (this is just modified example code from the ESP8266 library):
Upload that code to the Arduino and then open up serial monitor (the little magnifying glass on the top right). You should see it successfully connect to your wifi and then print the results of your get command, in our case an empty array and a timestamp from pubnub:
Setting up your Arduino
Now that we are connected to the internet, let’s set up a visual indicator of whether our code is working. I’m going to use a relay since it has a built in light indicator
The relay has 3 inputs:
- Power (+)
- Ground (-)
- Signal
Connect power to 5v/vcc and ground to one of the GND pins and the GPIO 5 for the signal pin. When GPIO 5 switches to “HIGH” the relay should light up, when it’s on “LOW” it should be dark.
Subscribing to PubNub channel
In order to receive commands from PubNub, you’ll need to subscribe to a channel and wait for Lambda to publish something to the same channel. Arduino has a useful built in “loop” function that predictably loops and executes any code in the function repeatedly, we’ll put our code into it to ping the channel repeatedly and see if anything has been published to it.
You can do this via PubNub’s REST API and do a GET request in the loop or use the PubNub Arduino library and modifying it slightly to make it compatible with ESP8266. In both cases we’ll need to parse the json result.
If you have a mega2560 or board with more than 2k bytes of memory, you can modify the example code provided by PubNub (just change the client to be esp8266) and use the aJson library. My Arduino didn’t have enough memory for it so I used the ArduinoJson library. You can see it pings the channel and then if it catches a relevant command it will execute:
You’ll need to configure the code with your wifi and PubNub keys.
Upload to your Arduino and open up serial monitor again, you should see it having successfully connected to your internet and be pinging the channel for any updates.
Now go into PubNub debug console, for the channel select “hello_world” and send a command just like what lambda would give:
You should see the command show up in your serial monitor:
If you don’t see it make sure you’re using the right keys and are on the right channel.
Now let’s test the whole thing. As before go to your Alexa testing tab and under “enter utterance” type in:
Turn on 5
Verify that it’s pushed to PubNub as above and that it shows up in lambda serial monitor:
and that the relay turns red!
Success!
3b) Raspberry Pi:
This tutorial is in python, I have one in nodejs coming with a longer tutorial on how to build a tea robot. Hope you find it useful!
Set up
Get a Raspberry Pi, make sure it has a WiFi card (buy a usb one if it doesn’t) and get connected to your WiFi network.
Optional: I’m going to use the same relay as in the arduino tutorial above to indicate whether the pin is on high or low. Connect the power (+) to 5v, ground to gnd and signal wire to pin 5
Next install the necessary Python libraries on your Raspberry Pi, type the following commands into the command line:
sudo pip install ‘pubnub>=4.0.13’
sudo apt-get install python-rpi.gpio
Code
Next, create a Python file called “gpio.py” with the following code:
In terminal run it via “python gpio.py” and you’ll see it say “Waiting for Commands”. Now go into your “service simulator” as above and tell it to turn on pin 5
You should see it show up in your terminal:
And see the relay turn on/off.
Success again!
Conclusion
Now that you’ve successfully issued your first voice command to your Raspberry Pi or Arduino, you can quickly iterate on this setup to perform far more elaborate tasks.
While I mentioned a few ideas earlier, I’d love to see what creative projects you come up with! If you can, please share videos and comments with me at alina@initialized.com!
