Probabilistic machine learning in 5 minutes, an intuition.

Maybe you have not heard about probabilistic machine learning, but as soon as Bayesian models will be implemented into production, it will be a critical, yet difficult, concept to know.

The reverend Thomas Bayes, mostly known by its Bayes (or Laplace) Theorem makes us rethink machine learning.

Introduction

Everyone is so happy with neural networks in production, they provide predictive power, and are trendy. However, their productions may not be the best explanation, or better, the unique explanation, for a plethora of different data problems. Remember, neural networks are essentially models that are parametrized by a tensor W of weights. I do not pretend to be here very technical and you may wonder why I am speaking about this, but please just keep reading and do not be afraid. The optimizer of the neural network, via the backpropagation algorithm, estimate some tensor values such that they minimize a loss function L(W) over a particular dataset D. Oh, life is very happy and this is easy… well.. you may be suffering from overfitting…

Overfitting emerges when we do not learn the tendency, but the spurious details of our particular sample that do not explain the tendency of the data. (Photo: https://www.analyticsvidhya.com/blog/2020/02/underfitting-overfitting-best-fitting-machine-learning/)

The problem with a plug-in approximation