Live Twitter Sentiment Analysis (With Deployment)

Abhishek Darekar
Nov 6, 2020 · 5 min read
Image for post
Image for post

This story will be divided into 4 parts :

  1. Connecting with Twitter API and extracting the data.
  2. Preprocessing the Data, and Using TextBlob for sentiment analysis.
  3. Create an API using Streamlit and Flask.
  4. Deploy the project on Heroku.
  • Our Goal will be to Create an API where the user will Enter a Topic, which we will search on Twitter and Extract tweets related to that Topic.
  • We will then do sentiment Analysis on the extracted tweets and classify them into Positive, Negative, Neutral.
  • Further, we will provide visualizations so the Data can be further analyzed by the user.
  • Here is a link to the Project i have Deployed, just so we are clear what we are working towards.

Link : https://twitter-sentiment-analysis07.herokuapp.com/

All the Steps we are going to discuss can be found here in the TwitterSentimentAnalysis.ipynb file. I will only be covering the important concepts here, the rest is just standard Python code.

Github Link: https://github.com/DarekarA/TwitterSentimentAnalysis_Public

1.Connecting with Twitter API and extracting the data.

  • For this you will need a Twitter Developers account, if you don't have one it is pretty easy to get, Just follow these steps :

Apply for a Twitter Developer Account.

https://www.extly.com/docs/autotweetng_joocial/tutorials/how-to-auto-post-from-joomla-to-twitter/apply-for-a-twitter-developer-account/

  • Once you get the account Just create a dummy app, and from that app we will get the necessary Keys & Tokens which we need for the API.
  • The 2 Steps for this are :

1. A Twitter app can be created via the Twitter app dashboard page with an approved developer account.

2. Generate access tokens on the “Keys and Tokens” tab in an app’s “Details” section within the Twitter app dashboard. Click the “Create” button in the “Access token & access token secret” section.

Image for post
Image for post
  • After the connection is established, we can query the Twitter Api and ask for the data we want.
  • We will have a DataFrame “df” ready to store this extracted data.
Image for post
Image for post
Fetch Data using tweepy.Cursor()
  • Using tweepy.Cursor() we will fetch the tweets for the Topic we selected, Every tweet will come with information such as username of the person who tweeted it, Likes/Retweets for that tweet, Location of the User and so on.
  • We can just use whichever attributes that interest us and store it in a DataFrame so we can further process it.

2. Preprocessing (Clean)the Data, and Use TextBlob for sentiment analysis.

Image for post
Image for post
Image from : https://dimensionless.in/
  • Like any other project, we will need to clean the data before we perform any analysis on this.
  • Cleaning stage will be straightforward , we will remove any tags like “@,#” . We will also remove Special characters and any links.
  • All this can be done using a simple Function
Image for post
Image for post
Function to clean all the extracted Tweets.
  • Now that the data is clean, we can use this data to analyze the sentiment using TextBlob.
Image for post
Image for post
  • TextBlob is a big library for processing textual data, however we are for now only interested in the “Polarity” score provided by TextBlob, Using this we can classify the tweets into Positive, Negative or Neutral.
  • TextBlob gives Polarity of a sentence. Polarity range is [-1,1].
  • -1=Negative
  • 0 =Neutral
  • 1 =Positive
Image for post
Image for post
Function to analyze the sentiments
  • Once we get the sentiments, we can analyze the data as usual.
  • Below are some examples of how we can analyze/visualize the data.
Image for post
Image for post
Overall Distribution of Sentiments.
Image for post
Image for post
Pie Chart for Different Sentiments.
Image for post
Image for post
Most frequently used Words from the tweets we extracted.
  • Similarly we can go ahead and do many such visualization and different Analysis based on the sentiments we derived from the tweets to get an idea about what the discussion on Twitter is happening about any topic we selected.

3.Create an API using Streamlit and Flask.

4.Deploy the project on Heroku.

  • In addition to our .py file, we will be needing a :

1.Procfile , 2.setup.sh , 3.Requirement.txt (in CMD Just type Pip Freeze to get the requirement file). All these files are available in the GitHub link.

  • 1. Update the procfile (this is starting point of the program)
    2. update the setup.sh file
    3. Dump all this files directly in Github Repository.(Not inside folder)
  • Then just go to the Heroku app and Link you GitHub, choose the correct directory and Deploy the Project .
  • And just like that your Project is ready to use for everyone!!!
  • Hope these steps help you in the process of Deploying the end-to-end project and it goes seamlessly. In case of any queries, any addition you feel we can do to this , do reach out to me!

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Sign up for Analytics Vidhya News Bytes

By Analytics Vidhya

Latest news from Analytics Vidhya on our Hackathons and some of our best articles! Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Abhishek Darekar

Written by

AI and Machine Learning Enthusiast.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Abhishek Darekar

Written by

AI and Machine Learning Enthusiast.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store