Sentiment Analysis, Use Cases and Challenges

With the rapid growth of the web and social media, access to customer reviews is much faster and easier than ever. Customer-generated content such as product reviews, social media posts, forums, blog posts, surveys, or chatbot data is just like a gold mine to reach customer opinions on products and services.

Many companies seek for scalable and cost-effective solutions to collect the gold mine in one channel and to analyze it. Sentiment analysis helps companies to generate fast, conscious, and strategic more informed marketing and development decisions.

In this article, we will provide a general view on sentiment analysis methods, common uses, and difficulties encountered in this analysis.

What is Sentiment Analysis, How Does It Work?

Sentiment analysis is a text analysis operation that focuses on analyzing and processing emotions on a text to extract the sentiment class. Sentiment analysis helps companies to understand how their products or services observed from the view of customers. This is generally achieved through the analysis of written or verbal feedback. Sentiment analysis helps companies in processing online dialogs and feedbacks on products, brands, or services to follow the insights of customers. There are several approaches to determine the type of sentiment that feedback contains.

Manual Operation

Human review is important especially for cases like understanding context, resolving ambiguity, or understanding irony. The language contains the history of society and keeps developing itself with new events in that society. Therefore, interpreting the meaning of a text that can change concerning context can only be done by a conscious mind. This situation makes usage of human annotators necessary in cases where the computer algorithms are weak. However, as can be expected, it leads to costly and sometimes inconsistent solutions. On top of that considering the rapid growth on the amount of textual data makes manual evaluation of data impossible.

Keyword Processing

Keyword processing can be considered as a sentiment analysis part of rule-based algorithms on NLP. Specific keywords are tagged with sentiment scores and those scores are considered to evaluate overall scores of longer texts. The overall sentiment score of a text is obtained by the weighted sum of sentiment scores of its keywords. This score leads to a sentiment class for the text. There are different approaches to deciding sentiment scores. For example, assigning human specialists to assign points is one of those approaches, however, this approach might lead to subjective judgments. Another approach is to keep counts or probabilities of keywords in texts. This approach will reveal more consistent results. There can be many other approaches for this task, in any of these approaches considering the context of keywords is the key. Otherwise, there will be certain mistakes in classification. For example, interpreting the term “high” without context may lead to inaccurate evaluations:

• My energy is so high that no one can upset my mood!
• The dollar rate closed this year too high, causing a decline on the day of purchase!

Natural Language Processing: (NLP)

NLP uses context, relations, and patterns to analyze text and extract sentiment values. NLP makes use of machine learning algorithms which include morpho-syntactic and semantic processing steps. These algorithms use given datasets and train a classifier model. The aim is to build a probabilistic model that will make use of features of the language, capture the context of the text, and interpret those to generalize its predictions. This pipeline will automatize the learning of language and task-dependent properties.

Artiwise Analytics uses machine learning and deep learning algorithms to process textual data and extract aspect-based sentiment classes instantly.

Types of Sentiment Analysis

Sentiment analysis can be initially defined as classifying a text as positive, negative, or neutral. However, in detail, it has different branches for different use cases. For example, while simply classifying a text into one of the 3 sentiment classes is enough for some cases, in some others it will be necessary to assign different sentiments into different parts of the same text. Considering different use cases of sentiment analysis, the most popular approaches in this field can be listed as follows:

Objectivity/Subjectivity Analysis:

Whether the text includes a subjective judgment or not is the first question to ask on a sentiment analysis task. The text might be objective, in which case there is no sentiment to evaluate. Objectivity/subjectivity analysis determines if there are subjective judgments in the text or not. It does not focus on the type of sentiment, but it focuses on the existence of subjective expression.

General Sentiment Analysis

General Sentiment analysis allows a single sentiment assignment to a given text. This approach used to be very popular, yet it lost reputation after observation of inaccurate/insufficient results on analyses done using this approach. This approach overlooks the fact that documents may contain more than one sentiment scores inside.

Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis assigns sentiment values to features or subjects in a text. It extracts different topics covered in a text and assigns sentiment values to each of these topics.

Aspect based sentiment analysis is the best approach for customer experience-oriented projects. It gives detailed information on which parts of your product do the customers use and on which parts they have satisfaction/dissatisfaction.

For example, I am pleased that the car has been delivered fast, but it would be great if you had not forgotten the brake disks.

This sentence includes multiple topics and multiple sentiment values. Therefore, if we only assign a sentiment score to the whole text, we will definitely loose information. Aspect-based sentiment analysis overcomes this problem and assigns following sentiment values to parts of this text:

1. Text: I am pleased that the car has been delivered fast Subject: Delivery Sentiment: Positive

2. Text: but it would be great if you had not forgotten the brake disks.

Subject: Spare Parts Sentiment: Negative

Fine-grained Sentiment Analysis:

Defines positive, neutral, or negative poles of documents including surveys and social media comments. It can give results both on the document base or aspect base. It defines local polarities and compares those internally to generate a global polarity score for each part. This enables classifying a positive comment as positive while classifying another positive comment as “very positive”. This kind of classification helps to detect edge cases.

Emotion Detection

This type of sentiment analysis covers more detailed sentiment types. It does not use standard types as positive, neutral, or negative; It uses classes like happiness, sadness, anger, disappointment, etc. Systems with emotion detection algorithms do not only analyze customer opinions as positive or negative, in addition to that, but they also analyze the mood of the customer and take actions considering the customers’ psychological state.

Use Cases and the Importance of Sentiment Analysis

By using the possibility to detect the polarity of user-defined assets and concepts, sentiment analysis provides a flexible tool for every scenario for consumers. It tracks whether the processed text is subjective or objective; also it looks for if the text has contained traces of irony on a global level. By doing that the sentiment analysis gives the user additional information about the reliability of the polarity obtained from emotion analysis. It can be implied easily customer voice, social media, chatbot, and survey data, which are the main source of real feelings and opinions because of their direct and spontaneous specialties. Scientific researchers, political parties, election periods to PR celebrities who want to follow public perception are the most current examples of using areas. Sentiment analysis gets much more powerful when used for the voice of the customer and voice of the employee. Business analysts, product managers, customer support teams, human resources, and workforce analysts and other stakeholders use emotion analysis to understand how customers and employees feel and why they feel about certain issues.

Social Media Monitoring

In the age of social media, a single viral review can burn the entire brand. With Artiwise Analytics, you can automatically analyze customer feedback from survey responses to social media conversations, get tips on what matters most from the perspective of users, empathize with your users and adapt your products or services to meet their needs.

Customer Voice

The emotional analysis helps workforce analysts and Human Resources (HR) managers listen to employee confusion at its source, understanding what employees are discussing and how they feel. HR teams use emotion analysis to proactively address painful points and increase morale through rich analysis of employee surveys, team messages, emails, and other communications.

Customer Feedback — NPS (Net Promoter Score)

NPS surveys are the most commonly used methods for getting an answer, feedback for basics questions like “Would you recommend this company, product, and/or service to a friend or competitors?” These kinds of surveys mostly use point scoring systems. With those points, companies classify customers in their part; promoter, natural, and detractor. These definition references are intended to identify the overall customer experience and find ways to theoretically upgrade to the full supporter that means a level where they will buy more, stay longer, or steer other customers. Quantitative survey data is easily collected and evaluated, but the next question in NPS surveys ask the reason for the score they gave customers. This triggers a series of open-ended responses that are much more difficult to analyze. However, sensitivity analysis allows these texts to be classified as positive and negative, giving more information about why customers give these scores.

Real-time sentiment analysis tools are driving forces on improving customer satisfaction. Analysts extract useful information by inspecting social media comments or news data on different scales. Customer support directors and social media leaders cover questionable cases before they go viral and lead these cases to product managers.

Difficulties in Sentiment Analysis

Machine learning/deep learning-based sentiment analysis approaches depend on the usage of big data. Therefore, a sufficient amount of data needed to build such a sentiment analysis system It is inevitable for the model to make an erroneous evaluation when there is not enough data to keep the system alive or when the system is not kept up to date. The sentiment value of a text can be influenced by features of the text.

In the piece of text, emotion analysis is influenced by the object, attribute, visionary, orientation and vision power elements.

For example; Telephone, Computer

Attribute: specific components and features of the object

Component example; touch screen

Feature example; size, processor speed

Person Opinion: Person/organization expressing emotion

To get complete, accurate and actionable output from a text, it is important to understand each of these five elements separately, but also how they work together to provide full context and feel.

Keyword processing usually depends on the sentiment of a single word; therefore, it fails to capture features of a text which affect the sentiment type.

NLP uses data analysis, machine learning, and deep learning technologies to overcome difficulties like language conveys. Grammatical nuances, misspellings, uncertainty, and regional differences can be listed as examples of these difficulties. Sometimes words can be used in ironic statements and the sentence means just the opposite of what you read. And in most cases, it is impossible to extract the true meaning without context knowledge. Dealing with this kind of sarcastic statement is one of the hardest problems in analyzing sentiment of a text along with cases like followings: Usage of contradicting words in a sentence, difficulty of extraction entities, referential ambiguities, etc…

How Does Artiwise Analytics Process Sentiment?

Artiwise Analytics real-time categorizes user-centered data using powerful machine learning/ deep learning algorithms and emotion detection using three sentiment classes. Data classified as positive, neutral, and negative sentiment scores makes it easy to create and interpret deep insight into customer opinions with various data charts on the report screens.

Artiwise Analytics helps to process big data with minimum latency, minimum cost, maximum accuracy, and maximum consistency. Instantly categorized data can be very useful to detect which topics need attention or improvement:

Decide

Different sentiment scores can help you determine ways to re-organize teams or to create original strategies.

Be One Step Ahead on Competition

There are strategic advantages to know how customers feel about rival companies. Sentiment analysis helps to foresee customer tendencies. And this will provide you a control data for comparing your company with others.

Manage Product Lifecycle

Sentiment analysis gives insight into how successful your product on the market, which parts of it should be improved, and which features worth investment.

Expand Customer Experience

Sentiment analysis provides a certain opportunity to understand customer focus, to fix issues customers detected, and to put more resources behind strong aspects of your product.

Get Personalized Notifications

Be notified about negative comments on your product before it turns into a crisis. Customers give feedback mostly on social media, get detailed information.

Empathize with Your Customer

Learn how your customers feel about your product or service. Take the customer experience to the top!

Artiwise Analytics is created with modern machine learning algorithms. It uses aspect-based sentiment analysis and extracts sentiment values for each topic in a text. It supports sentiment analysis in 6 different languages: Turkish, English, Arabic, German, French, and Spanish.

Apply here for a demo session on Artiwise Analytics.

Originally published at https://www.artiwise.com on Aug 14, 2020.