Building a Home Automation Alexa Skill with Pat
Introduction
The purpose of this blog is to document integrating Pat’s meaning API for a home automation application, in this case with lights. As a mobile and web developer working with the Pat team and with early access to the Natural Language Understanding (NLU) meaning API, I ran an exercise to build Pat into a small home automation application via Alexa. The following is the process I used and thoughts on the initiative.
Getting Started
The following is an inside look at creating Home Automation skills within Amazon Alexa. I’ve taken the time to source platforms and skill development to share my experiences with you. First off is how to get started with the app architecture.
It’s a pretty basic structure that follows the suggested architecture that Amazon recommends for creating Alexa skills. To handle the controlling of the lights, I decided to use the Philips Hue API.
There are many different NLP and machine learning (ML) platforms that you can use in creating an Alexa skill. In this instance, I leveraged Pat, a next generation natural language understanding meaning API for artificial intelligence offering meaning-as-a-service.
So, why use Pat?
- No training needed
You can start using Pat without having to do any training with the NLU. In the interest of quicker development time, this is very helpful as the training of an NLU would probably take the majority of the time in developing the app, which I wouldn’t want. I’d prefer it to work right out of the box so that I can focus on the home automation functionality.
- Scalable
Since Pat does not require training, it would be easy to expand the capabilities of the skill in the future. If I would want to support other devices or use cases, then I wouldn’t have to worry about the NLU side of things.
How to Create the Home Automation Skill
Step 1: Setting up the Alexa Skill
Creating an Alexa skill is fairly easy, you just need an Amazon developer account to be able to create the skill, visit https://developer.amazon.com. Instructions for creating the skill are fairly straightforward and Amazon’s own documentation offers a comprehensive guide on creating a skill.
If you’re using an NLU framework for setting up your skill intent, like I did, you can bypass Alexa’s own language processing. You’ll need to capture the entire command to be voiced out by the user, and send this to your NLU framework of choice. To do this, you can set-up the skill to have a single intent that has a list of random utterances that ideally capture all the voice commands that the user could possibly use.
Pro Tip: For random sentences, use a sentence generator like https://randomwordgenerator.com/sentence.php.
Step 2: Creating the Skill Service
When creating an Alexa skill, you need a cloud-based service that handles the requests for the skill, and Alexa will then route incoming requests to your service. When creating a service for a custom Alexa skill, you have the option to use AWS Lambda, but in this case, I used a self-hosted web service using Node.js.
a. Integrate Your NLU Framework
To handle the NLU layer of the skill service, I used Pat’s matching meaning service. You feed this service with the sentences that Alexa detects. Using this service will get you a breakdown of the sentence that looks something like this:
b. Process Responses
The JSON data that Pat responds with contains objects that you can then use for deciding which commands to send to the lights. Specifically:
- Undergoer — The Undergoer can be used to determine the object that is supposed to do the action in the sentence. Pat breaks out the arguments clearly to see ‘who did what to whom’. In the apps case that would be a specific light bulb that needs to respond to the commands that are given to Alexa.
- Nucleus — This can be treated as the type of command that Pat detects in a sentence. In the case of the image above, Pat identified “turn-on” as the Nucleus in the sentence “Can you please turn on the lights”. By taking the ‘definition’ which means ‘activate’ (see the logical structure) Pat unambiguously shows what the requested change is.
With these objects, Pat was able to detect that Undergoer is “the lights“, Nucleus is “turn-on”, from the sentence “Can you please turn on the lights”. These are the two items required to be tracked in the service. With these two in hand, it’s fairly easy to map out the values to commands or actions that are then passed to the Hue API.
In the future, to support more devices and actions it would just be a matter of handling the different role value (undergoer here) for the former and the Nucleus values for the latter.
c. Integrate the Hue API
For controlling lights in our home automation app, we decided to use Philips Hue API. Hue allowed us to control some basic functionality like the brightness and turning the light on/off. Philips provides a simple step-by-step guide on their website that is quite straight forward to follow as well. To test out your API, you’ll need at least a Hue-capable light bulb and a Philips hue bridge, which is used to control the light bulb.
`Step 3: Testing Your Service and Voila!
To ensure the service is working properly, you can use the built-in Service simulator in the Alexa developer console.
What’s Next
While the scope of the skill was limited to just light automation through the Hue API, you could certainly increase the range of devices to be supported and integrate other APIs. You can also leverage Pat for other platforms aside from Alexa, the most common use cases would include NLU for chatbots.
You can read the follow up article where I address questions regarding this article & pat.ai in general.
For any questions on our experience and on Pat’s meaning API, please contact us to discuss. And if you’d like to waitlist for Pat’s meaning API, visit pat.ai.
