Building a sentiment analysis bot with IBM Watson and Raspberry Pi
Updated on May 30, 2019:
Hey! I have updated this article after two years. Go to the Tutorials section on Slack API Docs for the latest article. Thank you!
Do you ever wonder how your teammates perceive your Slack chat messages? Maybe you sound heroic! Or unpleasant. Perhaps you project harsh undertones you haven’t even noticed!
Imagine a helpful robot that analyzes your written communications. Now — stop imagining, because I created this (physical!) bot that analyzes message tone using IBM Watson, “reading” the emotion of each posted message!
In this article, I’ll share how I created the bot.
First I’ll show you how to use the Events API to capture messages, send them to IBM Watson for analysis, and then post the result using chat.postMessage.
Then I will present a totally optional but fun exercise! We will port our bot to a Raspberry Pi and reflect emotions using colors produced by an RGB LED.
🐙🐱 The source code is available on GitHub.
Before Starting This Project
This project uses Node.js, so make sure Node.js is installed on your machine. You’ll also need an IBM Bluemix account and add the Watson Tone Analyzer. Your service credentials will be required later.
In addition to the software bot, if you want to build a physical Raspberry Pi bot that shows the results with color LEDs, you’ll need some affordable hack-friendly hardware. You can either buy the IBM TJBot Kit that includes the cardboard robot, or prepare separately:
- 1 Raspberry Pi 3 (If you are using Pi 2, you need a WiFi dongle too)
- 1 NeoPixel RGB LED
- 3 Female/female jumper wires
You need to install the latest Raspbian OS, connect to WiFi, and update and upgrade the system. See Raspberrypi.org for additional instruction.
Building a Sentiment Analysis Bot
OK, let’s build a Slack bot that reads messages and analyzes for emotional content!
Our bot’s workflow:
- Read each message on a channel
- Send the message to IBM Watson for examination
- If the likelihood of an emotion is above the given confidence threshold, post the most prominent emotion to the Slack channel
Creating and Configuring Your Slack App
First, let’s create a new app at https://api.slack.com/apps?new_app=1. Fill out the (1) App Name and (2) choose a slack team that you are allowed to develop apps on:
Click Create App when finished.
Next, you need to configure your App.
At Add features and functionality, enable “Bots”, “Event Subscription”, and “Permissions”. We will walk through them soon.
Also, you will need your API credentials you can find at the App Credentials section when you code. They identify your app to the Slack platform.
Setting up a Bot User
Click the Bots button under Add features and functionality, or the (1) Bot Users menu from the left navigation to set up a bot user. Then (2) give your bot a username you want:
Make sure to click the Save Changes every time you make a change in this configuration process.
Setting OAuth & Permissions
Click OAuth & Permissions from the left menu.
You will need the pre-generated OAuth Access Token, beginning with xoxb- during development. (Note: To distribute your bot to the rest of the world, you will need to set up an OAuth button, etc, however, this tutorial does not cover the process.)
Scroll down to Permission Scopes section to add the scopes you need to use for the bot. At the pulldown menu, choose bot:
Setting Event Subscriptions
To receive channel messages, we are going to use the Events API. Click Event Subscriptions. You need to enable the feature by setting the Request URL, but let’s skip this for now.
Scroll down to Subscribe to Bot Events. Click the Add Bot User Event button, and select message.channels:
Setting up your Request URL
During the development, I am using a temporary URL from ngrok, which serves my localhost to a public URL. Download ngrok for your OS, unzip it, and install it by moving the file to the appropriate directory, for example, if you are using Mac, use the command, mv ngrok /usr/local/bin. You can learn more about ngrok at Using ngrok to develop locally for Slack.
Once you install it, run it on terminal:
$ ngrok http 5000The ngrok tool generates a URL looking like http://618830b2.ngrok.io for your local server at localhost:5000. Copy the ngrok URL and paste it into the configuration setup, but not so fast! The URL must be validated, and in order to do that you’ll need to finally write some code. We will come back here later.
Running an Express Server
Express is a simple, modular web framework for node.js. We’ll use it to quickly create our Request URL webhook receiver.
First, set up your node application:
$ npm initAnd configure your application. Then install Express.JS and body-parser (for POST):
$ npm install express body-parser --saveOnce finished installing these modules, create a index.js file, and instantiate the express server, listening on port 5000. Since you’ve set ngrok to localhost:5000, you must use the same port!
'use strict';const express = require('express');
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));const server = app.listen(5000, () => {
console.log('Express server listening on port %d in %s mode', server.address().port, app.settings.env);});
Now, create HTTP POST route to handle the validation code. Let’s create an /events route. This will be triggered every time an event arrives, like the message.channels event we want from public Slack channels.
When you enter the webhook URL on your configuration page, Slack will send a request with a challenge parameter, so your endpoint must respond with the challenge value to complete the handshake.
app.post('/events', (req, res) => {
if (req.body.type === 'url_verification') {
res.send(req.body.challenge);
} // will implement the bot here ...
});
Now, let’s run this node script.
Then (1) go back to the Event Subscriptions, and (2) enter your ngrok URL with the route, for instance, http://618830b2.ngrok.io/evens. If everything works fine, the Request URL will verify, and you should see a green checkmark like this:
Handling event subscriptions
Let’s start coding your bot!
Go back to your index.js file and at the /events POST route you defined, we’ll receive and handle the message payload.
Within the request handler, you need to check the payload to see (1) if the token value matches with your verification token (which you can find at the App Credentials section of the Basic Information), and (2) if the type value is the event_callback, extract the message text to be analyzed by Watson later:
app.post('/events', (req, res) => {
let q = req.body;
// 1. To see if the request is coming from Slack if (q.token !== process.env.SLACK_VERIFICATION_TOKEN) {
res.sendStatus(400);
return;
} // 2. Events - get the message text else if (q.type === 'event_callback') {
if(!q.event.text) return;
analyzeTone(q.event); // sentiment analysis
}
});
If you just want to run the code locally, you may hard-code your token here. There are multiple ways to set your environment variables, but I usually set a .env file to include the variables. In the source code, you can take a look at the .env-test file to see how it is set.
Now, create a function, analyzeTone, which take the message text to be analyzed and some other info from the payload.
Using the Sentiment Analysis with Watson
Now, let’s use IBM Watson for the sentiment analysis, so get your Service credentials from your IBM Bluemix account ready!
Go to your Bluemix console, and choose Tone Analyzer from the catalog under Watson.
At the next screen, click Create.
Once you activate the Tone Analyzer, (1) click the Service Credentials from the left menu, and you should be able to obtain your username and password for this specific service by (2) clicking the little triangle by “View credentials”.
To use the Watson Node.js library, install the watson-developer-cloud module from npm:
$ npm install watson-developer-cloud --saveThen instantiate it with your credentials:
const watson = require('watson-developer-cloud');
let tone_analyzer = watson.tone_analyzer({
username: process.env.WATSON_TONE_ANALYSIS_USERNAME,
password: process.env.WATSON_TONE_ANALYSIS_PASSWORD,
version: 'v3',
version_date: '2016-05-19'
});Now, you sent the Slack message text to the Tone Analysis:
const confidencethreshold = 0.55;
tone_analyzer.tone({text: text}, (err, tone) => {
tone.document_tone.tone_categories.forEach((tonecategory) => {
if(tonecategory.category_id === 'emotion_tone'){
console.log(tonecategory.tones);
tonecategory.tones.forEach((emotion) => {
if(emotion.score >= confidencethreshold) {
postEmotion(emotion, ev)
}
})
}
})
});Watson can analyze a lot more from a given text, but for this project we are only using the emotion analysis. The results will be returned as a JSON that provides a hierarchical representation of the analysis. Each emotion has an index, so define the confidence threshold (let’s set it 0.55 for now. You can adjust it later!) and use only the emotion the exceeded the threshold value! Learn more about the watson-developer-cloud module on the GitHub repo.
Posting a Sentiment back on a Channel
Now, let’s create a postEmotion function and post the result back on the Slack channel!
To post a message, use chat.postMessage API. This call requires values for the token, channel, and text. The token should come from an authentication of a user with the OAuth, however, during the development, use the token that begins with xoxb- that you can find at the OAuth & Permissions section.
function postEmotion(emotion, ev) {
let message = 'feeling ' + emotion.tone_id;
let options = {
method: 'POST',
uri: 'https://slack.com/api/chat.postMessage',
form: {
token: 'xoxb-.....', // Your Slack OAuth token
channel: ev.channel,
text: message,
as_user: false,
username: 'Watson Bot'
}
};
// Use Request module to POST
request(options, (error, response, body) => {
if (error) { console.log(error) }
});
}Yipee, your bot should be working now! Let’s run your node code. Add the “sentiment_analysis” bot to your Slack channel, and try posting some emotional messages!
The code samples used here are simplified to fit in the tutorial, however, if you wish to include the teammates’ usernames in the result messages as seen in this screenshot, view the entire source code on the GitHub repo!
You can end the project right here, or proceed this tutorial for more fun with hardware!
Making the Bot into a Physical Bot with Raspberry Pi!
More bots are more fun! Let’s port this code to Raspberry Pi to make it as a physical bot!
Installing Node.js on Pi
You need to boot up your Pi with Raspbian, connect to Internet, and upgrade the system. If you are new to Raspberry Pi, I recommend to learn how to set up on RaspberryPi.org!
Then you can either directly work on the terminal on Pi, or SSH into the Pi from your computer, and install Node.js for Linux ARM:
$ curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -$ sudo apt-get install -y nodejs
Installing ngrok on Pi
A nice thing about ngrok is that it supports ARM, so you can use in on a Raspberry Pi too!
$ cd /tmp$ wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-arm.zip$ unzip ngrok*$ sudo mv ngrok /usr/bin/
SSH into the Pi to copy your index.js and all the module dependencies over to the Pi from your computer. Run the index.js, and now you should be able to tunnel requests to your Pi, just like you did before on more expensive computer:
$ ngrok http 5000This gives you a new ngrok forwarding URL, so don’t forget to update the Request URL at the Event Subscription on your Slack App setting page!
Try posting some message on Slack and see if it works.
Now, let’s work on hardware!
Wiring Up the Pi and an LED
Follow the diagram below and wire a Neopixel RGB LED. Find a flat side of the LED, and the leg closest to the flat side is a data-out (no wire), then ground, power, and data. Connect each leg to a corresponding pin:
Light Up an Emotion!
Now you programmatically light the LED up, in a color depending on the emotion.
First, install the rpi-ws281x-native via npm:
$ npm install rpi-ws281x-native --saveThen write the code to turn the LED on. Let’s define the colors too:
const ws281x = require('rpi-ws281x-native');
const NUM_LEDS = 1;
ws281x.init(NUM_LEDS);
const color = new Uint32Array(NUM_LEDS);
process.on('SIGINT', () => {
ws281x.reset();
process.nextTick(() => { process.exit(0); });
});const red = 0x00ff00;
const green = 0xff0000;
const blue = 0x0000ff;
const yellow = 0xffff00;
const purple = 0x00ffff;// Show a specific color for each emotion
function colorEmotion(emotion) {
if (emotion.tone_id === 'anger') {
setLED(red);
} else if(emotion.tone_id === 'joy') {
setLED(yellow);
} else if(emotion.tone_id === 'fear') {
setLED(purple);
} else if(emotion.tone_id === 'disgust') {
setLED(green);
} else if(emotion.tone_id === 'sadness') {
setLED(blue);
}
}// Set the LED to the given color value
function setLED(colorval){
color[0] = colorval ;
ws281x.render(color);
}
Now, call the colorEmotion function at where you are calling the postEmotion().
Run the Node code, type some message on Slack and see the LED lights up to match your feelings. Hooray!
I hope you enjoyed the tutorial! If you have questions and suggestions, please leave comments. Also, when you build an interesting apps, do let us know!
Learn More
- IBM Watson Developer Cloud
- IBM Watson Node SDK on GitHub
- IBM Research TJBot
- Raspberry Pi Teach, Learn and Make with Raspberry Pi
Related Slack API Documentation
- Building Slack apps
Slack Platform documentation - Events API
Slack Platform documentation - Bot Users
Slack Platform documentation - chat.postMessage
API method documentation
