Why intelligences fail?
These days, everyone has become an expert. Everyone knows about Covid-19, everyone has an opinion on what should have been done before, what should be done now, and what should be done later. I am not an virus expert, but I know a bit about Artificial Intelligence, so while I was a few days off, I had time to think about how (all) intelligence(s) fail.
Why it failed?
Black Swan
I will start with a pictured example everyone should know. Take the chart below representing the variation of a variable over time. How do you feel about it?
How would you expect the variable to evolve in the next days, after day 1000? Imagine the variable is somehow linked to your wealth, you would be very happy as it seems to always grow over time. The only question you might ask yourself is how much more it will grow? Which of the scenario A, B or C would most probably happen?
But this chart is taken from a now famous book from Nassim Nicholas Taleb named The Black Swan: The Impact of the Highly Improbable (and thanks to Alex Fleischer, colleague and friend, for recommending me this book). In fact, it represents the weight of a turkey over time; and day 1000 is Thanksgiving.
And here is what happens next.
On Thanksgiving the turkey gets eaten.
The importance of having some experience
Was this final so improbable as the title of the book suggests? It really depends from which point of view you look at it.
- From the turkey perspective, it has no experience of this process. Mostly all the data available to the turkey is included in the initial chart.
- But from an average american perspective who knows the variable is turkey’s weight, and day 1000 is Thanksgiving, and who had been eating turkey at Thanksgiving every year for the last 30 years, the conclusions may be different.
Note that all words in this last sentence are important. You need to know what the data is representing (variable is weight, and day 1000 is Thanksgiving), and you need to be an American citizen with experience. Most French citizen are ignoring what Thanksgiving means to a turkey. We can say that the “unprobableness” of an event is relative to the point of view. From the turkey point of view, Thanksgiving never happened, but for a US citizen it happens very regularly every year.
This situation is very well known by Machine Learning (ML) experts as the model training is always biased by the data used. The luggage packing example is my preferred example to illustrate this cognitive bias: I imagine my kid, who has learnt to pack luggage during 10 years of looking at family luggage packing being done for summer holiday, and who may get in serious trouble the first time he is going to ski with his friends. Ski holidays are his particular Thanksgiving.
Intelligence fails when experience is missing or cognitive biais exclude less probable events.
The importance of knowing what is behind the data
Now takes this new chart.
You might think it is the same. May be, or may be not. This one might represent the Dow Jones level over the last 1 000 days before Covid-19 epidemic. Or may be it represents the need for intensive care respirators over the last 1 000 days before the Covid-10 epidemic. Aren’t we like turkey in this case?
Intelligence fails when data is missing or not correctly labelled for interpretation.
So, before taking decisions, we would need to get and understand all the data. But in practice this is not always obvious, some data has to be extracted, created or predicted. I talked about the different types of data and the types of data science in a previous post.
Being able to extract additional data or knowledge from data is really one important meaning of Intelligence:
- the ability to acquire and apply knowledge and skills.
- the collection of information of military or political value.
We could have seen the Covid-19 coming. In fact, some previous French health Minister acted and made huge stocks of masks, for example, but others did not believe in the risk and reduced the stock.
What to do next?
One thing is to foresee the unprobable before it happens, and some other thing is to decide what to do.
The importance of the objective
When it comes to taking a decision, it is also important to know what is the objective. Coming back to the chart. It it represents you wealth, what happens on day 1000 is dramatic. The outcome is not great if you are the turkey, but it is much better if you are the one who celebrates Thanksgiving and eat the turkey!
Should we save people or should we save economy? The Covid-19 crisis may be dramatic for those who dies and their families and also for the economy. But as shown in some recent articles, it might be good for the planet. In real life, problems with a unique well identified objective do not exist. You need to improve quality and reduce cost. This is life.
Intelligence fails when objectives are unclear.
The need for experience
Imagine you are in a plane without pilot. You enter the cockpit. All the data is here. You have the altimeter, the speedometer, and all kinds of other indicators. Suppose that you understand what all these data are and what they mean. Will you be able to have the plane land correctly? No.
Even if you have lots (all?) precise data on a situation, this is enough. You need something more that is called experience. In Machine Learning, this experience is what we call a model. A model is created using lots of historical data. This is what we call training a model. Training data is what I would call Artificial Experience.
By definition, in a world with Highly Improbable events, many new situations happen for which we don’t have historical data. There is no one to learn from, and there is no time to learn, as decisions are needed now. Noone is more expert that anyone else in a situation that noone has faced before.
This is where you need something else that is called abstraction. You need to formulate the data and the problem in an abstract way and spend time on it. You can formulate your problem into a language, a syntax, a framework where you have learnt to solve problems. And this is a different way of thinking. All this has been described by Daniel Kahneman. in his book Thinking, Fast and Slow (thanks to Javier Lafuente, ex-colleague and friend who recommended me this book).
While fast thinking uses long term past experiences to very quickly proposes decision, which comes very handy sometimes, slow thinking will require more time as it corresponds to a longer analysis and fact checking of the outcomes of fast thinking. And efficient humans are those who better combine the two ways of thinking.
The need for algorithms
Decision optimization is at Machine Learning what Slow Thinking is at Fast Thinking.
Decision Optimization (a.k.a as Operation Research) is about modeling the problem into mathematical equations which can be solved by mathematical algorithms. This requires some work to extract and use the knowledge from the real life, and this is what O.R. experts do. For quite some time, it has been considered that this work was a pain and bottleneck. Not extracting the right constraints or the right formulations may lead to unapplicable solutions in real life. But we have seen above that M. L. experts might also require some significant effort to extract and use the right data into their models.
Intelligence fails when calculations are wrong.
Humans do make errors, in particular when they face lots of equations and calculations. We are not good at this. And we thought that automating the calculations would solve all our problems. OR was basically not meaningful before the era of computing. It did not even really exist.
Now computers and algorithms are here. But, while algorithms to solve DO problems are still improving in terms of performance, they still only provide the right optimal solution to the problems they are given. So what can fail in DO is the formulation of the constraints or the objectives.
Finite world and constraints
Let’s take one example of constraints we have all seen again and again these day: the health care system capacity.
As you have been told every day, the objective is to take the right decisions to keep the peak below the system capacity. Is the peak a given data? In fact not. Partially it is unknown and needs to be predicted, and partially it is impacted by our decisions. All the work of individuals is to use physical distancing and wash our hands, so that the spread of the epidemic is slower, and the peak happens later and is smaller. There is also a geographical dimension here, as the peak does not happen at the same time on all geographies and another way to improve our response is to transfer patients over different areas.
Same for the health care system capacity: is it really a hard constraint? No, as while some decisions are taken to try to reduce the peak, some other are taken to try to increase the capacity. In the absolute it is part of the decision problem.
Of course the more decisions, constraints and objectives one take into account, the harder the problem is to solve. So exactly as the ML expert will efficiently choose the right data to take into account for training, the RO expert will efficiently choose the right constraints to take into account in the formulation, and the right level of modelling between data an decisions.
Intelligence fails when some important constraint is not taken into account.
The risk of not executing
And this is not always easy to consider the right constraints. In the exact same way cognitive bias apply and makes one take into account the data he has, excluding the Black Swan data, it is very easy to ignore some constraints which will look obvious afterwards.
Bill Gates had told the world about the epidemic, but he was not heard. Now everyone is looking back and stating that things will not be the same. Sure, we might be ready for a new epidemic, but may be not for the next new Highly Probable event.
In that case, we did predict the evolution of climate and its consequences (GIEC calculations), we know what is causing these change, we know the constraints (what will happen if we go over 2,5 degrees). There is nothing Highly Unprobable here, ans still we appear to be failing.
Intelligence fails when we don’t act in front of evidence, when obvious decisions are not applied.
All intelligences are welcome
As very well explained in the Thinking: Fast and Slow book, all human success comes from the right mix of the two thinking. Most efficient humans are those who can make the best use of the Slow and the Fast thinking. And I would say the same for AI. It would be completely stupid to think that AI should be based on data and experience only, and it would be stupid to think that AI should be based on knowledge and equations only. We need both. Some are better in some cases, and in general best outcomes come from the right combination of both.
Among AI, ML has the hype now and a lot of people are somehow bashing the old fashioned slow DO thinking. I bet they will regret. My work consists of ensuring both technologies are available, separately or combined as part of the tools and platforms we develop such as Watson Studio or Cloud Pak for Data. Not only one can feed the other, but there are ways to better take into account predictions with probability distribution into decision optimization, for example using stochastic or robust optimization. You can look at some of my other posts to learn more about this, such as this introductory one on how to start combining ML and DO.
P.S. After I shared a draft of this post to Alex, he mentioned me this other post form IBM Research colleagues also talking about the analogy ML&DO and the book from Kahneman.
P.S.2 It is not commented here, but I want to mention another kind of artificial intelligence based on Business Rules. I talked about these 3 types in this post.
