How to Extract Tweets from Twitter in Python
The first step to obtain data is to set up the Twitter access from its developer platform with two methods:
- Obtaining Twitter API keys
- Connecting to the Twitter API
There are two ways by which you can connect to the Twitter API:
- Streaming API
- REST API
Streaming API gives access to real-time data, showing tweets as they are published (access to a sample of tweets).
REST API allows to query historical tweets (generally up to about a week), based on multiple criteria.
I will focus on extracting data/tweets via the REST API process.
Twitter API Keys:
The first necessary step is to have a Twitter account and get the necessary credentials on the Twitter developer platform to access the Twitter API by following these steps:
- Create a Twitter user account
- Log in with your Twitter user account at https://apps.Twitter.com/
- Click on developer.twitter.com
- You will land on the Twitter’s developer page. Next, under your corresponding ‘Name-XYZ’ tab search for the ‘Apps’ option and click on the same
- Click ‘Create an app’
- Fill out the form (not all details need to be filled, only the ‘required’ fields), agree to the terms and click on ‘Create’
- Go to the next page, click on the Keys and Access Tokens tab, and copy your API key and API secret key. Scroll down and click on Create Access token & access token secret, and copy your Access token, and Access token secret
Now that we are ready with the Twitter credentials required, let’s move on to the next stage, which is data extraction.
Step 1: Importing the necessary library and packages
Step 2: Defining parameters that will be used to establish connections with the twitter API
Step 3:
We have to select an endpoint (indicates where a particular resource can be accessed), which is represented by an URL.
Step 4:
Converting the results to json format and parsing the tweets using ‘BeautifulSoup’ parser
Voila!!! we have the recent 100 tweets extracted from Amazon-India’s Twitter handle
References:
