Machine learning, the antidote to “fat data”?

by Bertrand Maltaverne


Facts are key in business to decide and anticipate.

You cannot manage what you cannot measure.

This is a famous quote, actually attributed to various people. What is important is its meaning: fact-based decisions are crucial in business. The thing is that “gut feeling” is sometimes defined as a trait of great business leaders but I believe that this is simply the symptom that you cannot have all data that would be required and that a part of risk and intuition plays a role. Nevertheless, right decisions are and must be fact-based.


The haystack: big data = fat data?

As written above, people have a tendency to wanting to collect more and more data regarding a decision they have to make. This fits with the quote at the beginning of this article, and this also reflects a certain fear and uncertainty in a decision-making process. Finding the right balance to have what is enough is the secret!

Too much data is not always better!


In addition to the psychological need for more data that I described above, new technologies open new doors in terms of ability to collect more & more data. The Internet of Things, a.k.a. “IoT”, with its plethora of sensors greatly, enhances our capacity to measure and connect. The IoT comes with its load of data. As shown below, investments in IoT will be the highest in the manufacturing industry.

Source: http://uk.businessinsider.com/the-enterprise-internet-of-things-market-2015-7

Actually, this interest for always more data in manufacturing is no wonder. It’s a sector with an engineering culture: 6 Sigma, continuous improvements, DMAIC… The people and processes are data-driven.

Too much data may lead to:

  • Infobesity
  • Analysis paralysis

So, when people talk about big data, there is a risk that it’s actually fat data, data that brings no value and actually hinders the decision-making process!


The tool to find the needle: machine learning to the rescue?

But there is hope in the race for better decisions because of more data… And technology is the saviour. As stated above, “fat data” is never far away from a big data initiative but technology, because of the advance in computing power, can fix that. Especially of you had machine learning to the equation. Actually, with machine learning, the more data, the better. Thanks to the latest advances in artificial intelligence and programming, machines can learn to perform complex tasks, including data analysis. A good example:


The needle: predictive analytics!

Machine Learning + Big Data = Predictive analytics — source

This approach is very different from “business intelligence”, business intelligence is more past-oriented. Predictive analytics is about:

  • Predicting impact in the future of decisions made today
  • Anticipating the future to take actions now
  • Recognizing patterns helping forecasting

Machine Learning + Big Data = New knowledge

A very impressive example of how computers can surpass humans is the example given in the TED video below where computers help doctors diagnose cancer and were able to generate new knowledge by finding hidden trends and correlations:


Applications in Procurement?

Being someone working in Procurement, this is the next logical question.
Here are some answers:


If you enjoyed this piece, please scroll down and click the “recommend” or “share” button. Thanks!