Sentiment Analysis and Financial Services
Maybe you have heard about machine learning and you may have even heard about sentiment analysis. They might feel like buzz words to you, just like Superfoods they will come and go you think. And really, what has sentiment analysis ever done for you? How can we adapt it to help us with day-to-day financial services or even operational tasks? Let’s explore
A couple of weeks ago I found myself sitting in a Machine Learning Meetup Group in Bath and wondered the same. The topic for the evening was training a model to recognise a positive or negative film review. Using Sentiment Analysis, you utilise text learning to determine a writer’s attitude. It is already widely used by companies to detect positive and negative reviews or comments which enables them to engage with the writers directly. It is also the fuel that determines if an email is SPAM or not. In recent years, wider applications have emerged. Bloomberg, for example, are working to forecast market movements based on sentiments in news and social media (see https://www.bloomberg.com/company/announcements/trending-on-twitter-social-sentiment-analytics/).
But how, you might wonder, how can machine learning and sentiment help with all of that?
In the broadest of terms Machine Learning is supervised learning. You provide a framework for your programme and allow it to learn from existing data so that it can then go off and make decisions on its own.
In the workshop, we trained our algorithm to decide if a film review is positive or negative using polarity data which was compiled by Stanford University and consists of 5331 reviews for each sentiment. But first you need to train your bot. We used 2500 reviews of each set to define the vocabulary, a list of all unique words present within each data set. Next, a Natural Language Toolkit created groups (vectors) of negative and positive sets of words. And finally, you need to decide which algorithm you use to perform the calculations. We used a Naïve Bayes Classifier, which reviews each word and calculates its individual probability of being positive or negative. In case you are wondering, it is generally viewed as naive as it does not assume correlation, but independence of each word.
In a final step, we tested our bot on all the reviews and we managed to achieve an overall accuracy of 75% (In comparison, a pure rules-based approach using a human classified lexicon only yielded an overall accuracy of 55% for the same data).
For us in the world of finance, this can help to classify customer communication more effectively and order them in the priority of negative to positive which would ensure we handle angry and upset queries first or route them straight away to a specialist team. Or we use a chat bot which could be programmed to diffuse a situation before it escalates. It might even be better as we humans tend to justify our actions and take complaints personally whereas a bot could learn to handle this situation in the most efficient way based on previous experience. We could even target models or suggest certain investments for our clients based on previous sentiments and behaviour. And sentiment analysis could even lend a hand in financial crime and be utilised to detect fraudulent conduct as it would have access to all client communications and hence notice unusual sentiment on a broader scale.
At SECCL Technology I will be looking to train a chat bot as well and will keep you informed how we are getting on.