AiPet: GoPiGo + Alexa Voice API + Clarifai + Google Vision API

Now-a-days, there is so much off the shelf tech available in robotics and AI (Vision, Voice), I thought it would be cool to put it all together. There is no major engineering here; just making things work together. I saw Google Vision has a similar prototype to demo their API but they only explain the Vision API. Most good engineers can figure out how to put all of this together; so, this is primarily meant for product managers like me who enjoy tinkering but aren’t necessarily hardcore developers.

I’ve been building a smart “A.I.” pet using a raspberry-pi robot. The feature-set so far is too sparse to call it a pet but I’ve already named it…pi :). pi can see and interpret the environment using Google Vision API and Clarifai API. And it talks and follows commands using Amazon Alexa Voice API. So far it can do basic conversation (Alexa out-of-the-box using Amazon’s Raspberry-pi project), follow some commands (Alexa Custom Skill using sample deployment handler) and tell me with some accuracy whatever it’s looking at (Clarfai Python Client+ Alexa Custom Skill).

As you can see this is work in progress. I plan to continue experimenting. Some things on my list:

- connect pi to Calendar and Google Maps so it knows what’s the best time to wake me up for my first meeting at work; no more setting static alarms

- connect pi to Photos so it recognizes anyone who is tagged and call her/him by name

- give pi memory so it remember faces and conversations; the later one is challenging

I found most of the code online; so, ton of thanks to a number of engineers. I did some modifications to make the code work for me. I thought I share how I put it all together. The code is mostly python.

That’s it for now. I may do a follow-up post once I have more features working. If anyone is interesting in collaborating just ping me at uday.sandhar@gmail.com.