Billions and Billions of Dollars
Understanding US Congress #Jointsession using (artificially) intelligent information retrieval and sentiment analysis
I’m from India and I live almost on the other corner of the day as compared to the United States. So, when all the hype was going on in the US, like the academy awards and more recent joint session where President Trump was addressing the joint session, we were waking up for the next morning and pushing ourselves to brush our teeth and cursing the empty coffee machine.
The title of the post (“Billions and Billions”) is a very common twitter trend during #TrumpAddress as you know it on twitter. As I mentioned, I was up on a Monday morning still trying to make sure whether I was in a hangover when I saw the twitter app gave me a notification about the joint session. In fact, it was already 45 minutes into the address. So, I thought I would just see what happened on twitter later, and put twitter streaming API to work to get tweets than on and the REST API to crawl tweets that had already been made (just to tell the user, this API is generally not very generous in providing tweets, but on the day of #TrumpAddress, it just seemed to work like a charm). When I came back half an hour later, I had a corpus of 50,000 tweets from the REST API and 20,000 tweets from the streaming API. So, what to do now? Read all 70,000? No way!
OK, so how do I know what actually happened? Can I use some AI techniques to help me summarize the data about how are people reacting to the Joint Session address and discover the non-obvious? The recent advances in AI actually make sure that we have capabilities that can help us gain a big picture. One method is to run Karna-AI’s insights on the #TrumpAddress tweets to get an automated report like here we did for Bestbuy’s customer support. You can get a report like this made on any trending topic using by placing a request here.
However, this time the goal of the study was not getting the big picture like in the normal studies, but rather slicing and dicing tweets to get a feel of what is being talked about and discover nuggets of information. Please note that this is unstructured data we are talking about slicing and dicing here and we need a very efficient Swiss Army Knife to cut the clutter. We hence introduce SemSocial (Semantic + Social), a new semantic technology for Social Networks text. SemSocial can dynamically categorize twitter data into user defined categories instantly. SemSocial is expected not just to use words of category you define but also semantically related words (so for example tweets with “border” and “migrants” should be categorized with “immigration”). SemSocial is a variation of the technology that we developed for ParallelDots’ news recommendation engine earlier, but is way faster to train and retrieve information from (almost instant), due to availability of new technology of embeddings generation and nearest neighbor retrieval.
The first thing I was curious to know about was what all was being talked on twitter with#TrumpAddress hashtag about law and law enforcement. I simply define a category “law, enforcement” and get automatically categorized tweets and their sentiments using our twitter sentiment analysis algorithm. Also we remove all the re-tweets (or keep just first re-tweet if we don’t have the original tweet) as that would just give repetitive output in search. Let’s look at the search results:
This is it, pure original content tweeted related to “law, enforcement” we could get using Twitter’s APIs. The still duplicate tweets you see are same texts tweeted out with different links (copied pasted tweets basically). If I think of it, I was expecting some hard steps of Trump to come up about law and order. I discovered about VOICE (Victims of Immigrant Crime Engagement) from these tweets. You would notice that words law and enforcement do not occur in the tweets describing VOICE, but our search engine can still fetch them.
Let’s also find out what was going on about President Trump’s “billions and billions of dollars” of investment and his renewed focus on American jobs and business. Let’s see results for queries “business, taxes”, “new jobs” and “investment dollars”.
It’s mostly what I was expecting regarding tax cuts and all, the only new thing I discovered here was the Harley Davidson issue, which I was unaware of. Please note that the algorithm could dig it out itself without me giving any inputs, I just had no clue.
The tweets related to “new jobs” are more or less what I was expecting, got to know about a good initiative, job for disabled.
“Investment Dollars” category had the highest amount of relevant stuff. Lot of people complaining about and some people supporting Trump’s plans are visible here. We can see a lot of negativity from Trump himself complaining about current losses and people politically differing from him. Let’s verify that by quickly using our sentiment analysis capabilities.
So yes, the sentiment we see in the text verifies here as well. We can now quickly export the sentiment graph and use it anywhere.
Do you like the new feature? Please let us know. You can always request for an insight/study here.
Karna.ai is a division of ParallelDots. Karna is a social media marketing research platform. We collect and analyse millions of mentions from News, Twitter, Facebook and Instagram and deliver AI driven in-depth insights through automated reports and custom analysis.We would love to hear from you. You can also drop us an email at contact@paralleldots.com.
