Human thinking process and machine learning models in Decision Making

Thinking about how our mind thinks is one of my favorite pastimes. I have always been fascinated by the workings of the brain, or psychology as they call it scientifically. The quest I began, to learn more about this led to a whole new world of rediscovery in me and my perspectives, it has answers to all my questions on our thoughts, intuitions, and the innumerable biases we are prone to fall for leading up to critical decisions that make or break our life. This knowledge I have imbibed is only a speck of information available out there, but in this post, I have tried to sum up one interesting case of a cross over of psychology and a primitive machine learning algorithm. Like every mechanism of our thinking, this idea is also very simple, it’s just that it’s implicit in our actions and thoughts, that we often fail to recognize the roots obscured in what we are doing. Okay, at this point I feel like this is a good introduction to this post, but I also feel that I might have been carried away by my validation bias that says this is good enough, not thinking too much about what might have gone wrong.

Thinking about thinking

Human beings are hardwired to think and react spontaneously to everything we see. This reaction is reflexive and hardly demands any attention, it is taken care of by our mind without coming to our notice, and then there are some things that you think hard about, like choosing a movie to watch, that you exhaust all your options and weigh the pros and cons. This is a conscious effort, we are aware of the process and hence consumes more energy.
There are a few basic elements that control everything that we think about.

• Most of our decisions are taken by the fast brain, which operates
  automatically and quickly, with little or no effort, and no sense of voluntary control.
• The slow and thinking brain, which allocates attention to the effortful mental activities that demand it, including complex computations. The operations of this mechanism are often associated with the subjective experience of agency, choice, and concentration.
• The priming effect is also an important player in our decisions, we are biased towards the most recent experience or activity when we make a choice in the present. This is illustrated by a simple activity, given a fill in the blank word like SO_P, you are prone to think of the word “SOUP”, if you recently had a meal or thought about it, whereas you would think of the word “SOAP” in case you just came out of the bath.
• The WYSIATI effect is the mechanism that tricks our brain to think that “what you see is all there is” to the story and ignore all the other details. This leads us into many traps often, leading us into thinking of only selected attributes of an entity rather than a holistic view.

Let’s get some expert advice on this topic

How often do we consult expert opinion on some things we think we cannot decide upon, very likely that we would want to get in touch with a financial assistant to advice on what stocks to invest in, or a shrewd analyst on what trends we see in the future of an idea or a company.

But what we fail to think about is whether the field is itself predictable?

more often than not, we are overshadowed by the person or their experience that we don’t consider the fact that the field in itself is uncertain and has a lot of factors that might affect the decisions of these experts, no matter how experienced they are.

The case of financial advisors

A few psychologists studied the results and success rates of the top ten financial advisors of one of the best US economy firms. They took the data from the past 8 years about these top advisors and their trends, recommendations, and clientele.
There was an interesting pattern that emerged.

• No advisor was the top recommender for two or more consecutive years, the case was always that one year they trump and that feat is followed by years of failure or sub-optimal performance.
• When this information was presented to these firms, they accepted it without any surprise. This means the whole industry is based on a flawed principle.

This can be explained by the illusion of validation we often fall prey to.

The illusion of Validity describes the tendency to overrate our ability to make accurate predictions, and interpret data subjectively such that in strengthens our predictions.

Can a machine predict better than an expert?

The next experiment that was carried out was to try and let a machine learning model do a prediction that an expert would do and the answer was surprisingly positive. A rudimentary machine learning algorithm was proven to be more accurate than an expert by Robin Dawes of the University of Oregon in his famous paper “The Robust Beauty of Improper Linear Models in Decision Making”.

The premise of the problem.

This paper talks about using a primitive machine learning algorithm for predictions and replace the experts in the field with the same.
It talks about linear models in multiple variables to achieve this cause.

A proper linear model is characterized by a simple equation of the form

r = ax + by + cz

The variable ‘r’ represents the result of the equation, based on three parameters x, y, and z weighed by a, b and c respectively. The weights we are talking about is obtained by a statistical optimization or training and fine-tuning to tailor the results we are expecting.

This is a statistically proven algorithm and known to work on a multitude of problems. and there would be no doubt that this algorithm provided with enough training would outperform an expert, no doubts!

How about an improper linear model?

An improper linear model is the one in which,

the weights of the predictor variables are obtained by some nonoptimal method; for example, they may be obtained on the basis of intuition, derived from simulating a clinical judge’s predictions, or set to be equal.

what if I say that even these sub-optimal models outperform an expert? Yeah, this approach, in fact, was applied on a diverse set of problems, and was proven to be better than the decisions of the experts!

Graduate Acceptance decision problem

This paper emphasizes the case of predicting the acceptance rate of a graduate student admission based on a set of numerical attributes like GPA, GRE score, etc.,
The paper proves that even an improper linear model with unit weight, that means just applying equal weights to every variable and obtaining the result and deciding based on that is better than an expert.
Another improper method that this paper talks about is called “bootstrapping”

The process is to build a proper linear model of an expert’s judgments about an outcome criterion and then to use that linear model in place of the judge.

Why is bootstrapping so interesting?

The use of linear models is a crude representation of judges, by which I mean that the judges’ psychological processes did not involve computing an implicit or explicit weighted average of input variables, but that it could be simulated by such a weighting.
In essence, it works because every judge by his shrewd experience would develop an implicit equation and weight it according to his training implicitly, he does not imagine a mathematical representation with numbers per se, but his mind implicitly does it for him and we can model his brain using a linear formula. Now, if we successfully transfer this representation into an equation, we have a model that performs very similarly to an expert. yay!

How is this bootstrapped model better than an expert from which it was derived from in the first place?

This is the most interesting question this experiment answers. Every expert is influenced by two components in his judgment, the first one is the implicit policy he follows to make a decision, which is trained by his experience and is based on the numerical inputs he gets. The second component is the other contextual and extraneous facts like his biases, priming effect, fast brain, and the illusion of validity he has, which also can weigh into his perspective of a candidate he is evaluating.
For example, he might be biased towards students from one region due to it education level and this might play a role in his acceptance over another equally talented student from a not so reputed region or university.

Bootstrapping extracts the first component of a judge and provides us with a model-free of human biases explained above!

This is precisely why it outperforms the expert.
This was proven by assessing the first year GPAs of students who had applied for a particular university, predicted by an expert versus the prediction made by a bootstrapped model.
But sadly, bound to a lot of social and technical reasons, this is often not practiced in real life.

A deeper look into why this happens also reveals the fact that human beings are so vulnerable to our context that it has a profound impact on our decisions leading to inconsistency. Simple things can have a positive influence on our decisions, that we are more positive after a cold breeze on a hot day or having a sweet dessert in our choices than before. We are also confused if the number of parameters involved in a problem is high, that’s why we take a lot of time in deciding to buy a phone rather than a guitar of the same cost because we don't have to worry about tens of specifications.
We can try to be more aware of these factors and think more about our choices and not be affected by inherent biases.

References —
Thinking fast and slow, Daniel Kahneman.
The Robust Beauty of Improper Linear Models in Decision Making, Robyn Dawes, University of Oregon.