Context Retrieval — Why Sentiment Analysis is old
India witnessed the grand power of democracy in the recently concluded general elections. While it was an emphatic victory for the voters, delivering a decisive mandate for the incoming government, it was also a key piece of event for the observers in the analytics industry.
The war-rooms of the political parties were filled with analysts crunching numbers, delivering personalized emails and SMS, creating region-tailored speeches for the leaders and constantly tracking people’s mood on social media.
One such graphic caught my eye ; Sentiment Analysis Tracker. Many if these graphics were being constantly updated on news websites like ET. It read something like this, with multiple donut charts attached
People Talking about Party A: 123
Positive Sentiment : 80
It means that out of 123 people mentions on social media about Party A, 80 are inclined positively. What about the rest 43? What were they cribbing about?
A recent proof-of-concept for a client gave me an opportunity to mine their customer feedback. Being unstructured textual data, I employed NLTK module in Python to do n-grams, keyword extraction, POS tagging, entity recognition and collocations. Most of the customers were complaining about unfair company policies, billing and salesperson’s misguidance to name a few. Before I went any further, it was clear to me that Sentiment Analysis (SA)would hold little business value. So what next? Context, of course !
Given the large amount of textual data floating around in the vicinity, it is crucial that we develop a core understanding of the customer. SA tells me the ‘what’, but doesn’t answer the ‘why’. For most of the firms, answering ‘why’ is more critical to the bottomline. Let me explain;
Customer A: “ The agent defrauded me. He gave me a rental plan with higher value while a lower value rental plan was available.”
Customer B: ” The company is a huge cheat. They mishandled my billing and now I have to pay a huge amount. Ba****ds.“
Sentiment analysis would turn up -1 for each. Keyword extraction would return me “agent defrauded” ; “Rental plan” ; “billing”. Not much of a insight, it it?
However, if I analyse content via a tool like GATE , I might understand that agents are mis-selling products or billing engine needs further fine-tuning. If such complaints keep recurring against a single agent, maybe it’s time to fire him. Similarly, I might find billing being spoken of in different context, like overcharging, late payment cycles or customer oversight.
So, what next can be done?
1. Mine the text to find recurring contexts like these. Extract the keywords from them.
2. Machine Learning : develop algorithms by adding these keywords to a dictionary with context and raise a flag if a customer feedback matches them.
3. Keep adding to the dictionary and refining it overtime.
This would make for an extremely intelligent customer service engine. Imagine getting immediate and pointed resolutions to your complaint than a drab auto-generated “no-reply” email. This will not only help provide solutions to the customer once such a flag is raised, but also repair the processes and logistics of the firm in time. One way to avoid social media nightmares.
I have never been a big fan of Sentiment Analysis per se. Many still maintain that it is an evolving subject. Great work has been done by the likes of StanfordNLP and Senseforth, but I am increasingly convinced that identifying context and focused information retrieval would help create intelligent engines valuable to the business. In this age where customer service is often neglected, this could be the trick which wins over new customers and retains the old.
Image sourced from:
