Sentiment Analysis on Elections 2016

Back in late September / early October, a new feature was introduced to our Election 16 app called “Sentiment Analysis”.

This feature monitored election related tweets and determined whether a tweet was pro-Clinton or pro-Trump tweet. It used a combination of “Sentiment Analysis” and some custom logic to determine the proclivity for a tweet.

“Sentiment Analysis” is a field of Artificial Intelligence (AI) / Natural Language Processing that determines whether a piece of text has positive or negative sentiment. Our custom logic then correlated that to whether the sentiment was for Trump or Clinton. The categorization results were aggregated to draw a chart of where the sentiment was tilting.

This is a form of “automatic polling”. Anytime someone posted an election related tweet, they were implicitly stating their support for a candidate. Without running a manual poll, the app was basically running a continuous automatic poll using tweets as the input and using AI to determine the proclivity.

The app did not allow users to go and see historical sentiment but only showed that day’s aggregated sentiment. We are now presenting the historical data here.

The chart below represents the daily sentiment data since we introduced this feature. Only Trump and Clinton were considered and not independents. The X-Axis represents time and Y-Axis represents percent sentiment for each candidate for that day.

As you can see, till before mid-October, the blue & red lines fluctuate around the 50% mark. Somewhere around mid-October, the sentiment completely shifted towards Trump and that did not change till Election Day. During the third debate the realtime sentiment (not daily) had swung sharply towards Clinton (https://twitter.com/PinaryIO/status/788937757338931200) but post the debate it again settled into a pro-Trump sentiment.

Key Dates:

Debate 1: Sept 26, 2016

VP Debate: Oct 4, 2016

Debate 2: Oct 9, 2016

Debate 3: Oct 19, 2016

Election Day: Nov 8, 2016

*Please ignore data for Oct 25. That is an anomaly due to a bug in the code.

For some time after the third debate, I had started to doubt the validity of this feature because its results (pro-Trump sentiment) were contradicting almost every poll that various media outlets were publishing indicating Clinton would win. I would sit at the end of the day for quite a few days and manually validate the results. But they seemed to be correct. Now, the Election results have also validated the results of this feature.

by Aashish (twitter.com/@aashishpatil)

Founder, Pinary.IO

UPDATE (Nov 12, 9:41 PST):

I haven’t been able to correlate what happened around mid-October for the sentiment to completely tilt towards Trump and stop fluctuating. If anyone reading has thoughts, please share.