Sentiment Analysis on Tweets with Node.js

Photo by Carlos Muza on Unsplash

I will guide you to make a Node.js app that crawls tweets from Twitter and calculates a keyword’s sentiment analysis trend in last 24 hours. I will make the code to be generic to use any keywords but in the sample code, I will use “Bitcoin” as the keyword.

Beware that these settings may or may not work if you are using different versions of packages. I used Node.js v10.16.0 for running this.

Prerequisites

• Node.js
  Get Node.js v10: https://nodejs.org
• MongoDB
  Get MongoDB v4: https://www.mongodb.com/download-center/community
• Twitter API
  Sign up and register an app to get consumer_key, consumer_secret, access_token and access_token_secret.
  https://developer.twitter.com

Let’s start! 🚀

You need create a directory, mkdir tweet-sentiment && cd tweet-sentiment and run npm init -y on it. It will give you a package.json file like this.

Now it is time to install the needed dependencies.

npm install --save axios@0.19.0 mongodb@3.3.0-beta2 sentiment@5.0.1 twit@2.2.11

What are all these packages? 😫

Axios: Promise based HTTP client
MongoDB: MongoDB native Node.js driver
Sentiment: AFINN-based sentiment analysis for Node.js
Twit: Twitter API Client for node

Tweet crawler

In order to run sentiment analysis on tweets, you need to crawl the tweets in specific keyword and store them in a database.

In this case, you will use twit library to grab those tweets and store them in MongoDB using mongodb.

Note that the score is normalized to either 1 (positive) and -1 (negative), because you may not want to weigh specific tweets’ sentiment analysis heavier than other tweets. In here, every tweets are weighed equally.

Tweet crawler runner

You already wrote the tweet crawler. Now it is time for writing the runner for it. In this code, you will create a keyword register that runs every N seconds and some other codes for calculating the overall score index per keywords.

With these two files, you can run node runner.js for gathering and keeping the data for last 24 hours.

REST APIs for the analysis data

You will need to create two APIs. One for retrieving tweets with sentiment analysis data and the other one for getting score of the keyword.

Note that getDB is called for every endpoints. This is done this way, because in serverless environment, the database connection will not be guaranteed to last.

I introduced how to integrate sentiment library, twitter API and MongoDB altogether to build REST APIs of sentiment analysis tweet data of keywords.