Indigo Airline’s Twitter fiasco — Sentiment classification gone wrong
Indigo airline’s (leading private airline in India) Twitter reply to a customer’s tweet last year gained lot of bad publicity for the airline. A disgruntled customer had tweeted about misplaced baggage, but it was a sarcastic tweet thanking them. Indigo’s reply was to thank the customer. This went viral before Indigo deleted the tweet.
For an airline that enjoys the largest market share in India domestic air travel and claims to have the highest on-time performance, they could have done without this social media faux pas. Especially, since this came soon after couple of other PR nightmares.
Indigo is not alone in this. There are other similar faux pas by B2C enterprises, most recently by Delta airlines.
NLP & Sentiment Analysis
A bot has obviously responded to this customer tweet. And, the bot has classified the tweet as a Positive feedback (wrongly!). Detecting sarcasm in text is not easy, especially the ones from Twitter with the limits on characters.
Here are some suggestions to keep in mind while building a Sentiment Analysis model.
- Build a custom Twitter specific model, not a generic one, as Twitter has few things unique, like limits in characters (of course with higher limits now), that forces the tweeter to adopt a different style of writing
- Typically Sarcasm is detected by markable contrast of sentiment in a statement — for example, starting with a positive sentiment and ending with a negative one. In this case, there is no marked negative sentiment except the “same time” in context to the two other entities.
- Typically punctuation marks, quotes, interjection words are stripped out during pre-processing. These are especially useful to identify the correct sentiment, especially sarcastic tweets
- Entity extraction is key — while typically 2-gram extraction works better in most cases, for Twitter analysis, better to go for either 2-grams or 3-grams
- Better approach may be to extract 1-gram, 2-grams, 3-grams, 4-grams that occur immediately after a positive sentiment phrase
- Parts of speech tagging provides the easiest way to identify Twitter sentiment. While Sarcasm is not easy to detect with plain POS tagging, the hashtags provide a way to identify sentiment — many folks handle hashtags during pre-processing; it is important to have hashtags, and tokenize them
- Negation of a phrase changes the sentiment of a tweet (example: not happy with the customer service). This is straight-forward and can be handled thru’ feature extraction and tagging
- Lookup dictionaries can be built with common phrases, slangs, acronyms etc. and looked up to standardize objects during pre-processing
- Emoticon / Smiley dictionaries can be looked up is one of the most powerful ways of detecting sarcasm in a tweet
- Using continuous scalar classes to arrive at Sentiment (- to + scale with threshold to classify), rather than discrete classes (positive, neutral, neutral)
- There are publicly available Twitter and online reviews data set that can be used to train the model — some of them also have classifications for Sarcasm, Emoticons, Hashtags etc.
- Dependency trees can be built and sentiment tagged for each token. Key is to tag appropriate sentiment to a parent — any negative child token can classify the parent token with a negative sentiment (especially true higher up in the order)
- For a more complex model, Context based weights can be applied considering all the attributes of a tweet, including replies. This can be used as Supervised learning while training the model.
The intent of this article was to provide few ways of capturing or identifying the correct sentiment of a tweet. There are scripts available to help with many of these things, but of course will need custom code and effort to get a good Semi-Supervised model that can work well.
