How-to-Use Machine Learning for Buying Behavior Prediction: A Case Study on Sales Prospecting
A quick and practical read on how ML algorithms can be used to model and predict human buying behavior.
Human behavior and economic models
During Christmas break, I started reading a book called ‘Misbehaving: The making of Behavioral economics’. The author, Richard H. Thaler — winner of Noble prize for economics in 2017, is considered one of the pioneers of behavioral economics. In his book, he gives various examples to show how humans do not behave according to economic models, here is one example:
‘Normal people do not behave like in economical theory. They do not optimize’.
Perhaps that is why no economic model has been successfully built to predict human behavior.
So what is our buying behavior
No one wants to be sold but everyones wants to buy.
Most of our buying decisions are not based on well-defined logic. Emotions, trust, communication skills, culture, and intuition plays a big role in our buying decisions.
Machine Learning and Human buying behavior
The most common approach taken by many ‘AI-based’ sales startups is to identify the next buyer by mining internet data. They look at what people are talking about in social media and then identify those who are searching for a given product or service. However, as pointed out in my article ‘Want to grow sales? Stop cold emailing. Start prospecting.’, people who are already actively looking online are not the best potential buyers (or prospects) to sell to.
Let’s see how top salespeople identify a prospect.
Top salespeople identify a prospect before he or she goes out and announces publicly that he/she is looking for a product or service. They build relations and identify the needs of people, often before the prospects may start looking for a solution.
Can ML algorithms identify the needs of prospects without meeting prospects?
Although humans do not follow a well-defined logic, we do have some repeated patterns. We often buy the same things, behave in a similar way and follow similar intuitions. So if we can learn the buyer’s pattern, we may be able to identify the next buyer too!
When we look at ML algorithms, Neural networks are one of the most widely used ML algorithms these days. One of the main reason of having widespread use of Neural Networks is because it can create an approximation of any function. The approximation is based on data, which it learns or is trained with. So neural nets are able to learn similar responses for similar inputs.
A detailed explanation is beyond the scope of this article but if you are interested to know more about Neural networks, you can read here and here.
How can Neural Networks be used for Sales prospecting (i.e. identifying new customers for your product/service)?
I have pointed out what constitutes a good prospect and sales process in two of my previous articles, click here and here to read more details.
The biggest problem that most New Sales Development Representatives face are: a) identifying a good prospect and b) Building a customized process and pipeline suitable for the prospect.
Note: New vs Old buyer
I must note here that the buying behavior (and sales process) for new and old customers are different. In this article, I will focus on New customers — namely called New Sales development for B2B customers. In sales term it is called Sales Prospecting.
Can machines be taught to behave like a top Salesperson? Let’s give a shot.
Part I: Identifying your prospect and creating a persona
I ask the following four questions to identify who are ideal prospects (taken from the book ‘New Sales Simplified’ by Mike Weinberg)
• Who are your best customers?
• Why did they become customers?
• Why do they still buy from you?
• Why do prospects choose you over other similar products?
The goal is to identify common features among successful and unsuccessful prospects. Normally this is done manually and intuitively.
If we had to solve the same problem via Machine Learning we need to use Neural Network Classifier.
Classification can be defined as the grouping of things by shared features, characteristics, and qualities or if you will simply dropping things into corresponding buckets, you could, for instance, classify the following geometric shapes based on their similarity. [Reference]
Step 1: Feature extraction
Based on the four questions mentioned above, we try to extract relevant features from answers to the questions. Here is an example of such a feature,
Who is your best customer: Customer size, Decision maker, Growth last year
Why they became customers: Location, First reference (personal contact, content marketing etc), Product features(Feature 1, Feature 2)
Why they still buy: Customer service, Location, Product features
Why they choose us over others: First reference, Product features(Feature 1, Feature 2), Location
Step 2: Labeling data
Label the data based on which of the leads took the least amount of time to covert, medium time to convert, maximum time to convert and did not convert.
Step 3: Training Neural Network
Once labeled, we will use supervised learning algorithm to train a standard Neural Network Classifier.
Step 4: Testing Neural Network
In this phase, you test how good the model is with the rest of the test data.
Step 4: Executing the Neural Network on new data
Once trained any new input with the data will be able to classify into good and bad output. Thus we can input either a person or company data and the Neural network will be able to classify.
Figure 1 (below): Neural network classifier
Part II: Creating a customized Sales process and pipeline
Once you know who can be a good/medium/bad prospect you want to create a customized process for that particular prospect. Top salespeople use intuition and experience to create such a process.
There are two potential algorithms that can be used for this. Long Short-term Memory (LSTM) and Reinforcement Learning.
Option 1: Using LSTM
A sales process can be seen as a set of actions done over time. The current action is dependent on what has been done before and what has been the response.
LSTM networks are perfect for that. These are part of the broader class of neural networks called Recurrent Neural Network (RNN).
One of the appeals of RNNs is the idea that they might be able to connect previous information to the present task [Understanding LSTM Networks].
As you can see in Figure 2(below), RNN is a series of connected Neural networks. Picture from here.
However, RNNs suffer from something called Vanishing Gradient problem. Learning is limited within a region of Neural networks and thus RNNs are not able to learn long-term dependencies. LSTM solves that.
Option 2: Reinforcement learning
Another interesting Machine Learning algorithm is Reinforcement Learning (RL). Reinforcement learning depicts the human way of learning. It is a learning based on real-time feedback and not via training data.
The learning algorithm learns best actions based on rewards and punishments it receives after executing an action in the real world. Figure 3 (below) shows a basic structure on how reinforcement learning works.
Now that we discussed how ML can help to identify new sales prospect, let us take it a step further.
ML to build personalized sales processes.
A sales process is a systematic approach involving a series of steps that enables a sales force to close more deals, increase margins and make more sales through referrals [NASP].
Rather than having a fixed sales process, the best salespeople build personalized and buyer focussed processes.
What action to take at a given point of time is decided based on what is the current state of the sales, past experiences of successes with similar clients and what is the goal of the current action.
If we had to model and learn a sales process, Recurrent Neural Networks (RNNs) seem to be the obvious candidate. One of the appeals of RNNs is the idea that they might be able to connect previous information to the present task [Understanding LSTM Networks].
As you can see in Figure 1(below), RNN is a series of connected Neural networks.
However, RNN suffers from something called Vanishing Gradient problem. Learning is limited within a region of Neural networks and thus RNNs are not able to learn long-term dependencies. If you want to learn more about the problem, here is a good article.
LSTM is a type of RNN but does not have the problem of Vanishing gradient. This article gives a very good overview of how LSTM works but briefly to explain:
Cell-state: This acts as the memory of the LSTM network.
Gates: There are three gates in each network. Forget gate, Input gate, and Output gate. Forget gate is used to forget a value from the cell-state. Input gate may add something into the cell state and updates the cell-state. Output gate is responsible for the output from that network.
The figure below shows Cell state and Gates.
Coming back to our problem of using LSTM to model sales process, imagine during a sales process based on a conversation with a company the salesperson learns about the priorities of the buyer.
Modeling that in an LSTM cell may look like below.
The salesperson might find new information and has to forget the old information and update the cell-state with the new information.
In this way, the LSTM network can be trained with tens of thousands of past sales process data and build an approximation model of the process. See below.
Final conclusion
I do not believe machines can replace salespeople. Machines will aid salespeople and can convert an average salesperson into a top salesperson.
An obvious question many of you will ask, do you need to build all of these algorithms yourself? Obviously not. There are libraries like Tensorflow, Keras, etc which you can use to train your model.
One common problem is that your model is as good as your data. That is why what most data scientist do is basically filter out the good data from the bad data. That’s a challenge!
Enjoyed this article? Here is another one.
If you want to receive updates on our AI Challenges, get expert interviews, and practical tips to boost your AI skills, subscribe to our monthly newsletter.
We are also on Facebook, LinkedIn, and Twitter.
