Sentiment analysis and opinion mining

Our everyday lives are full of decisions. And while some of these decisions, like buying a TV or a car, are large and important, others appear to be of lesser importance.

However, all of them can impact our happiness and well-being. That is why we look for help in making decisions. We consult with friends and family, we research products and services, or follow advice from experts. With the amount of information on the Internet today however it is difficult to get right sources that provide correct information and opinions.

This is where online platforms can help.

The number of things to buy and maintain in our homes is off the charts. This explosion in the number of products on offer was paralleled by an even steeper growth in online opinions regarding them — reviews, recommendations, likes, dislikes and more. Checking a product’s customer satisfaction and success rate is much easier nowadays.

Opinion mining is a process of automatically collecting and analyzing user reviews from social media, forums, product review/rating sites and elsewhere on Internet in order to understand users’ interests and needs, manage their brand reputation and improve their products.

A lot of consumers complain about products/services and leave bad reviews on various review sites, social media channels and blogs. It’s not unusual to see negative comments by people who are inexperienced with a product or service, but there are positive reviews as well. So, how can businesses know which complaints are worth addressing and which should be ignored?

Sentiment analysis is one of the answers to this.

Sentiment analysis is a technology which allows, based on a text, to determine whether the author of the text holds positive, negative or neutral opinion about some entity (e.g. restaurant, movie).

Sentiment analysis identifies opinions in texts. It is commonly used to predict a subjective value of the text with respect to the target, e.g. if a review is positive or negative. It can also be used for parsing questions, suggestions or opinionated texts or for distinguishing emotions or attitudes. Sentiment analysis models need appropriate pre-processing. Pre-processing is an important part of data pipeline.

One application is also in stock and crypto market research, where we can collect millions of tweets per day, determine its sentiment and then compute e.g. crypto bitcoin sentiment on an hourly basis or on lower latencies. See example chart for Bitcoin:

Following Bing Lu, we can introduce a more formal definition of an opinion:

Opinion is a set (e,a,s,h,t), where e is an entity, a is aspect of entity e, s is the sentiment on aspect a of entity e, h is the opinion holder and t denotes the time when holder h expressed this opinion.

Most type of sentiment analysis focus on so-called sentiment polarities, which can be positive, negative, neutral or in some cases divided in more categories, e.g. 1 star — very negative, 2 stars — negative, 3 stars — neutral, 4 — positive and 5 — very positive.

Sentiment values are usually categorical, known as sentiment polarities and are denoted as positive, negative and neutral.

Based on this definition of opinion, we can rephrase the main goal of the sentiment analysis:

The objective of sentiment analysis is to determine all opinions (e,a,s,h,t) in a given text.

Sentiment analysis can be applied on images as well. For this purpose we have built free online ocr software for extracting text from images with help from colleague from mathe online.

Sentiment classification can also be used to evaluate sentiment of technologies.

We are currently implementing it with our technologies usage platform to identify technologies that have highest positive sentiment written about them and identify verticals where this is most prevalent.

Here is e.g. chart showing the relative usage of Shopify across IAB1 verticals:

In our next article, we will look at types of sentiment analysis and prepare a code for Aspect Based Sentiment Analysis.