A Robot Befriends Classic Monsters Using Watson APIs — Part 4

Teaching TJBot to Interact a Few Classic Monsters

First all, good job making it to the final stretch 👏👏👏 — you’re almost done! In the last part of this tutorial series, we’ll put everything we’ve learned into TJBot.

The code running on TJBot is here. If you completed my Build a ChatBot That Cares tutorials, this should look very familiar.

Recap

Just a quick recap — if you started at the very beginning of this series, look upon the following list and admire at all that you’ve achieved. If you happen to have landed on this page first, here’s the entire list of tutorials and components required before tackling this last part.

Required List of Tutorials

• Build a Chatbot That Cares Part 1
• Build a Chatbot That Cares Part 2
• A Robot Befriends Classic Monsters Using Watson APIs Part 1: Custom Visual Classifiers in Python
• A Robot Befriends Classic Monsters Using Watson APIs Part 2: Custom Visual Classifiers in NodeJS
• A Robot Befriends Classic Monsters Using Watson APIs Part 3: Using Raspberry Pi Camera
• A Robot Befriends Classic Monsters Using Watson APIs Part 4: Putting It All Together (You are here)

Required List of Watson Services (Credentials)

• Watson Speech to Text
• Watson Tone Analyzer
• Watson Conversation
• Watson Text To Speech
• Watson Visual Recognition

Required List of Software and Hardware

1. Node.js version 4.4.5+ (I’m using 7.2.0)
2. TJBot
3. Raspberry Pi. I’m using the latest Raspberry Pi 3.
4. HDMI display
5. USB keyboard
6. USB mouse
7. USB microphone
8. Speaker with 3.5mm audio jack
9. Raspberry Pi Camera Module v2
10. Dracula, Frakenstein, the Wolfman, and the Mummy plush set

Step-By-Step Tutorial

Step 0. What You Need

1. See Recap section above.

Step 1. Starting the Camera

Ok that was a lot of parts. But if you completed them all, the rest should be fairly simple.

Important Note: Build a Chatbot That Cares Part 2 has all the instructions on setting up Raspberry Pi, including installing NodeJS and the necessary audio libraries. The rest will not work if you haven’t completed that tutorial.

I start the Raspberry Pi camera on line 5 of the following gisst, right when the script is run. This isn’t the most optimal solution (I’ll explain in the How To Improve section in the end), but it makes the code easier to understand. Since the purpose of this tutorial is to inform, I’ve decided to keep the code as simple as possible.

Step 2. Performing Character Recognition

You need to do is to wrap the recognizeCharacter function in a promise to make sure you actually classify the captured image and identify the character before building the response via Watson Conversation.

I directly return the name of the character that I’d like TJBot to pronounce.

Then simply run the code after the image has been saved to file.

Step 3. Passing Character Name to Watson Conversation

The returned monster name is passed to Watson Conversation via the context variable (see line 3).

Inside the Watson Conversation Tool, we use the $ to extract the value of the character and make it part of the response.

Since the character name is part of the Conversation response, TJBot will automatically speak its name as part of his response.

That’s It!

You just got TJBot talking to classic monsters! Put different characters in front of TJBot to make sure he’s recognizing the them correctly.

How To Improve

One drawback of this system is that the Raspberry Pi camera takes a long time to turn on and then shuts off right after taking a photo. I decided to go with this approach because it makes the code easiest to understand.

Unfortunately this means we have to run the script every time we need to recognize a new character. A better system is to turn the camera on in the beginning, and have it take pictures at regular short intervals. Your code will then simply use the most recent photo for character recognition. Ideally, you’d also delete older photos so you won’t run out of memory after awhile.

Final Thoughts

My dear readers, we’re finally at the end of our journey with TJBot. If you’ve followed along since the beginning, here’s some mad props from Watson — 🤖🙌. You should now be very familiar with many of Watson’s APIs, along with TJBot itself. It’s time to go build some cute robots of your own! Remember to come back and share what you’ve done, I’d love to see them.

Finally, a big shoutout to Nick Bourdakos for helping me throughout this project.

As always, if you have any questions, feel free to reach out at joshzheng@us.ibm.com, connect with me on LinkedIn, or follow me here on Medium.