I will guide you to make a Node.js app that crawls tweets from Twitter and calculates a keyword’s sentiment analysis trend in last 24 hours. I will make the code to be generic to use any keywords but in the sample code, I will use “Bitcoin” as the keyword.
Beware that these settings may or may not work if you are using different versions of packages. I used Node.js v10.16.0 for running this.
Get Node.js v10: https://nodejs.org
Get MongoDB v4: https://www.mongodb.com/download-center/community
- Twitter API
Sign up and register an app to get
Let’s start! 🚀
You need create a directory,
mkdir tweet-sentiment && cd tweet-sentiment and run
npm init -y on it. It will give you a
package.json file like this.
Now it is time to install the needed dependencies.
npm install --save email@example.com firstname.lastname@example.org email@example.com firstname.lastname@example.org
What are all these packages? 😫
Axios: Promise based HTTP client
MongoDB: MongoDB native Node.js driver
Sentiment: AFINN-based sentiment analysis for Node.js
Twit: Twitter API Client for node
In order to run sentiment analysis on tweets, you need to crawl the tweets in specific keyword and store them in a database.
In this case, you will use
twit library to grab those tweets and store them in MongoDB using
Note that the
score is normalized to either 1 (positive) and -1 (negative), because you may not want to weigh specific tweets’ sentiment analysis heavier than other tweets. In here, every tweets are weighed equally.
Tweet crawler runner
You already wrote the tweet crawler. Now it is time for writing the runner for it. In this code, you will create a keyword register that runs every N seconds and some other codes for calculating the overall score index per keywords.
With these two files, you can run
node runner.js for gathering and keeping the data for last 24 hours.
REST APIs for the analysis data
You will need to create two APIs. One for retrieving tweets with sentiment analysis data and the other one for getting score of the keyword.
getDB is called for every endpoints. This is done this way, because in serverless environment, the database connection will not be guaranteed to last.
I introduced how to integrate sentiment library, twitter API and MongoDB altogether to build REST APIs of sentiment analysis tweet data of keywords.