Natural Language Processing and Sentiment Analysis

Do you understand human emotion? Me neither. But machines can! We’re going to start with something you may already be familiar with. It’s called “Natural Language Processing,” and it’s pretty much what it sounds like — computers learning to understand what we say when we use our “natural” language. But we’re not going to stop there. We’re going to get into something really trippy, called “Sentiment Analysis.” That’s pretty much what it sounds like too, but it’s incredible when you think about it — computers learning not just to understand what we say, but what we mean!

Natural Language Processing

Let’s back up a bit. As any programmer knows, there is a big difference between the way humans communicate with one another, and the way we “talk” with computers. When writing programs, we have to use very careful syntax and structure, but when talking with other people, we take a lot of liberties. We make short sentences. We make longer sentences, we layer in extra meaning. We make multiple sentences with the same meaning. We find multiple ways to say the same thing. You get the idea. It’s complicated!

As artificial intelligence finds its way into more and more of our devices and tasks, it becomes critically important for us to be able to communicate with computers in the language we’re familiar with. We can always ask programmers to write more programs, but we can’t ask consumers to learn to write code just to ask Siri for the weather. Consumers have to be able to speak to computers in their “natural” language.

Sentiment Analysis

Natural language processing is only half the battle though. Human communication isn’t just words and their explicit meanings. Human communication is nuanced and complex. You can tell based on the way a friend asks you a question whether they’re bored, angry, or curious. You can tell based on word choice and punctuation whether a customer is getting exasperated, even in a completely text-based chat. You can read a review for a book and understand whether the reviewer liked or disliked it even if they never directly say so.

That’s because as humans, we’re very good at gauging sentiment. Sentiment is like a combination of tone of voice, word choice, and writing style all rolled into one. For computers to truly understand the way humans communicate every day, they need to understand more than just the objective definitions of the words; they need to understand our sentiments. They need to understand what we really mean.

Deep Learning

Deep learning can do that! By pairing lots of examples of natural language with labels about positivity or negativity (or any other spectrum we want to gauge), we can develop agents that can learn to understand the sentiments underlying new messages. What makes this even more powerful is that the popularity of internet forums (and the rising popularity of speech-based virtual assistants) give us massive databases to use to teach these agents, and once they learn, they can be set loose on new problems.

Imagine, for example, you’re a student in an online class, and you’re struggling with a problem. You post about it on the class forum. An AI agent performing sentiment analysis would be able to not only identify objectively what topic you’re struggling with, but also how frustrated or discouraged you are, and tailor its feedback to that sentiment. This is already happening, because the technology is already here.

Applications of Sentiment Analysis

There are so many applications of this technology. Imagine you work in media, and you write an agent that can learn what people like and dislike about movies by crawling the thousands of reviews already posted online. Imagine you work in investing, and you write an agent to assist with predicting the stock market that can evaluate the optimism of thousands of people talking about the stock in forums. Imagine you work in politics, and you write an agent to gauge the sentiment of people reacting to a speech on Twitter to gauge exactly what they like and don’t like. The implications are incredible!

With deep learning, this is all possible, and what’s more, it can even be done in real-time. You could build an interface where a speaker could see live analysis of their speech as they give it, and tailor it to those messages that are resonating most heavily. You could build an autonomous investing agent that makes its own investments based on a constant incoming stream of attitudes towards different stocks. You could build a classroom assistant that alerts a teacher in real time when the class has grown curious or discouraged.

I could go on forever, and that’s the most amazing thing about deep learning. Sure the technology is incredible, but it’s when you start thinking about the applications of this technology that things get really crazy. Once you master deep learning, it’s up to you to decide how you’re going to change the world with it. So, what are you waiting for? Join the revolution.