Sentiment analysis with KNIME and Azure Cognitive Services

Continuing my series on how to leverage the various Machine Learning and AI services from Amazon, Azure and Google, previously we used Amazon’s Rekognition and Python to perform image recognition with KNIME.

This time we’ll use Azure’s Cognitive Services with KNIME, leveraging their REST API. It’s incredibly easy to combine KNIME with these services in just a few nodes.

Rather than having to custom code solutions, I can, in just a few minutes build a workflow or integrate into my existing workflows a comprehensive sentiment analysis service. More impressive, not a single line of code will be written.

Combining Twitter data and Azure Cognitive Services for Sentiment Analysis

We’ll be leveraging KNIME’s built in Twitter functionality, along with Microsoft Azure’s Cognitive Services.

Before we start, you’ll need;

  1. a Twitter Developer Account (Free)
  2. a Microsoft Azure Account (Free for up to 5,000 requests)
  3. KNIME Analytics Platform

Make sure you’ve written down your from your Twitter developer account the API key, API secret, Access token and Access token secret.

You’ll also need your Azure Cognitive Services subscription key that we’ll use with the Azure REST API.

Creating your Azure Cognitive Service

When you login to your Azure Portal. Navigate to Cognitive Services and we’ll create a new service for KNIME.

  1. Click add and search for the Text Analytics service.
  1. Click Create to provision your service giving it a name, location and resource group. You may need to create a new Resource group if this is your first Azure service.
  2. Once created, navigate to the Quick Start section under Resource Management where you can find your web API key and API endpoint. Save these as you’ll need them in your workflow.
You can find your keys and endpoint within the Quick start section of the service

The KNIME Workflow

The KNIME workflow to combine Tweets and Azure’s Cognitive Services Text Analytics API

You can download the KNIME workflow to follow along here;

https://www.dropbox.com/s/439kigylzehgd7m/Sentiment%20with%20Azure%20and%20Twitter.knar.knwf?dl=0

The Workflow comprises of 3 parts;

  1. Pulling the data from Twitter and wrapping it into a JSON format compatible with the Cognitive Services API.
  2. Submitting that request via a POST Request.
  3. Taking the output and combining it with the Tweets metadata into a single table, colour coded and ranked by sentiment.

As an input, Azure is expecting the format;

{
"documents": [
{
"id": "1",
"text": "I loved the meal"
},
{
"id": "2",
"text": "I left hungry and unhappy"
},
{
"id": "3",
"text": "The service was excellent"
}
]
}

KNIME includes excellent Twitter nodes (if you don’t have them, you can download them from KNIME extensions) this will allow you to quickly and easily connect to Twitter and download tweets. By using the Twitter Search node we can search for particular terms.

We can then restructure the response from Twitter into a form that Azure is expecting. The Constant Value Column and JSON Row Combiner nodes wrap the response with the documents elements and this is then submitted via the POST Request.

KNIME’s POST Request node makes it ridiculously easy to interact with REST services. You’ll need to grab the respective URL for your region, here in Australia the URL is;

https://australiaeast.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment

We can leave the Authentication blank as we’ll be adding a couple of Request Headers.

We need to add for the Header Key;

Content-Type

and Header Value;

application/json

And another Header Key;

Ocp-Apim-Subscription-Key

The Header Value for the Subscription Key will be the key provided as part of your Azure Cognitive Services you created.

You’ll need to update the Oct-Apim-Subscription-Key with your API key

If you’re using my KNIME Workflow as a guide, make sure to update the Twitter API Connector node with your Twitter API Key, API Secret, Access token and Access token secret.

Add your Twitter developer credentials to the Twitter API Connector

You can modify this workflow to submit any text data you choose as long as there is an ID column and a text column.

Presenting the data

Sentiment analysis of tweets ranked by score and colour coded presented on KNIME Server

Once created you can use a Table View node to display the information in an interactive table, sorted by sentiment . This can be distributed to PR and Social Media teams for action, improving customer service.

To really supercharge your KNIME deployment and make this service truly accessible, you can use KNIME Server’s Web Portal to create an interactive online sentiment service for your social media team allowing them to refresh reports, submit their own twitter queries or provide alerting so your team can jump on issues.

As always, any questions, please feel free to message me on Twitter or LinkedIn.