Harnessing X’s API & Python: Sentiment Analysis for Stock Market Insights. Enhance Your Stock Portfolio with Python & X

Code like a Geek

In the fast-paced world of stock trading, information is king. With the rise of social media, millions of investors and enthusiasts share their opinions and insights on platforms like X (formerly known as Twitter), making it a goldmine of real-time sentiment that can significantly impact stock prices. However, manually sifting through countless tweets to gauge market sentiment is impractical. Enter the power of Python — a versatile programming language that, when combined with natural language processing (NLP) techniques, can automate the extraction and analysis of X sentiment related to key stock tickers such as “MSFT” (Microsoft) and “GOOGL” (Alphabet Inc.).

This tutorial aims to guide you through the process of leveraging Python to scrape X for tweets mentioning specific stock tickers, perform sentiment analysis on these tweets, and interpret the results to gain insights into the potential market movements of these stocks. Whether you’re a seasoned trader looking to incorporate social media sentiment into your trading strategy, a data scientist seeking to apply NLP techniques to financial data, or a Python enthusiast curious about the applications of your coding skills, this tutorial has something for you.

We’ll start by setting up our development environment and obtaining access to the X API. Next, we’ll dive into collecting tweets using the powerful Tweepy library, followed by performing sentiment analysis with TextBlob. Finally, we’ll discuss how to analyze and interpret these sentiments to inform your investment decisions.

Join us on this exciting journey to unlock the potential of social media sentiment in stock market analysis, and take your trading strategy to the next level with Python.

Step 1: Setting Up

You’ll need access to the X API v2, which requires an approved developer account. Once you have that, create a project and get your API keys (Bearer Token).

Install Necessary Libraries:

• The following steps assumes you have already setup your Python environment and have the pip module installed. Now, by running cmd as Administrator, enter the following command:

pip install tweepy textblob pandas

Brief explanation of the libraries:

• tweepy: For interacting with the X API.
• textblob: For performing sentiment analysis.
• pandas: For data manipulation.

Step 2: Collect Tweets

Use Tweepy to collect recent tweets mentioning the stock tickers.

import tweepy
import pandas as pd
from textblob import TextBlob

# X (formerly Twitter) API credentials
bearer_token = 'YOUR_BEARER_TOKEN'

# Initialize Tweepy
client = tweepy.Client(bearer_token)

# Function to fetch tweets
def fetch_tweets(query, max_results=100):
    tweets = client.search_recent_tweets(query=query, max_results=max_results, tweet_fields=['text'])
    return [tweet.text for tweet in tweets.data]

# Example usage
msft_tweets = fetch_tweets("MSFT", max_results=50)
googl_tweets = fetch_tweets("GOOGL", max_results=50)

Step 3: Sentiment Analysis

Perform sentiment analysis on the collected tweets using TextBlob.

# Function to analyze sentiment
def analyze_sentiment(tweets):
    sentiments = [TextBlob(tweet).sentiment.polarity for tweet in tweets]
    return pd.DataFrame({'Tweet': tweets, 'Sentiment': sentiments})

# Analyze sentiment
msft_sentiments = analyze_sentiment(msft_tweets)
googl_sentiments = analyze_sentiment(googl_tweets)

print(msft_sentiments.head())
print(googl_sentiments.head())

Step 4: Analyzing Results

You can aggregate the sentiment scores to get an overall sentiment for each ticker. A positive score indicates a positive sentiment, while a negative score indicates a negative sentiment.

# Function to summarize sentiments
def summarize_sentiments(sentiments_df):
    avg_sentiment = sentiments_df['Sentiment'].mean()
    return avg_sentiment

# Summarize sentiments
msft_avg_sentiment = summarize_sentiments(msft_sentiments)
googl_avg_sentiment = summarize_sentiments(googl_sentiments)

print(f"Average sentiment for MSFT: {msft_avg_sentiment}")
print(f"Average sentiment for GOOGL: {googl_avg_sentiment}")

The above snippets should be saved into one file, name it whatever (or I did, I named it “x_sentiment_analysis.py”), but must end with the file extension “.py”:

import tweepy
import pandas as pd
from textblob import TextBlob

# X (formerly Twitter) API credentials
bearer_token = 'YOUR_BEARER_TOKEN'

# Initialize Tweepy
client = tweepy.Client(bearer_token)

# Function to fetch tweets
def fetch_tweets(query, max_results=100):
    tweets = client.search_recent_tweets(query=query, max_results=max_results, tweet_fields=['text'])
    return [tweet.text for tweet in tweets.data]

# Example usage
msft_tweets = fetch_tweets("MSFT", max_results=50)
googl_tweets = fetch_tweets("GOOGL", max_results=50)

# Function to analyze sentiment
def analyze_sentiment(tweets):
    sentiments = [TextBlob(tweet).sentiment.polarity for tweet in tweets]
    return pd.DataFrame({'Tweet': tweets, 'Sentiment': sentiments})

# Analyze sentiment
msft_sentiments = analyze_sentiment(msft_tweets)
googl_sentiments = analyze_sentiment(googl_tweets)

print(msft_sentiments.head())
print(googl_sentiments.head())

# Function to summarize sentiments
def summarize_sentiments(sentiments_df):
    avg_sentiment = sentiments_df['Sentiment'].mean()
    return avg_sentiment

# Summarize sentiments
msft_avg_sentiment = summarize_sentiments(msft_sentiments)
googl_avg_sentiment = summarize_sentiments(googl_sentiments)

print(f"Average sentiment for MSFT: {msft_avg_sentiment}")
print(f"Average sentiment for GOOGL: {googl_avg_sentiment}")

5. Run the Script

• Open your terminal and navigate to the directory where you saved the Python script. You can use the cd command to change directories. For example:

cd path/to/your/script

Run the script by typing:

python x_sentiment_analysis.py

Below is a sample output run on my machine:

Step 6: Next Steps

• Correlate the sentiment analysis with stock price movements.
• The script will print the average sentiment scores for the specified stock tickers based on the recent tweets it analyzed.
• Review the output to gain insights into the general sentiment (positive, neutral, or negative) towards these stocks on Twitter.
• Enhance the sentiment analysis model by training it on financial lexicons.
• Use streaming data for real-time analysis.

Troubleshooting Common Issues

• Authentication Error: Make sure your Bearer Token is correct and has not been revoked.
• Rate Limit Exceeded: Twitter API has rate limits. If you encounter this error, try to run your script later or apply for elevated access through the Twitter Developer Portal.
• Module Not Found: Ensure all required libraries (tweepy, textblob, pandas) are installed correctly in your Python environment.

Other Important Considerations:

• Always respect the X API’s rate limits. The author of this article is using the “Basic” subscription, it is unclear as of today whether the “Free” version plan offers the same capabilities. Please do your independent research and visit X’s Developer Portal page for up-to-date plans: https://developer.twitter.com/
• Ensure your use of the X API complies with their terms of service, especially regarding data storage and privacy.
• Sentiment analysis accuracy can vary; consider using more advanced models or combining multiple models for better results.

This project structure provides a basic framework for sentiment analysis on stock-related tweets. By adapting and expanding upon this foundation, you can develop more sophisticated trading strategies or market analysis tools.

I hope you enjoyed this tutorial! Please feel free to comment and provide feedback. Thanks!