Whatever happened to Data Mining?
The concept of Data Mining has been increasingly displaced by Machine Learning. So why is that?
You don’t hear much about Data Mining these days, so I pulled up a quick Google Trends analysis to compare usage over the last 5 years. The trends are quite compelling, and show that since August 2015 the concept of Data Mining has been increasingly displaced by Machine Learning. So why is that?
Mining is slow and inefficient
I am descended from generations of coal miners. Life in the Lancashire pits of the 19th century was hard, there was little mechanisation and the output was meagre.
Miners need to sift through a lot of dirt
When searching for high value minerals such as gold or diamond mines produce huge amounts of waste. Those bewhiskered gold prospectors had to sift through a lot of dirt to find any nuggets.
Data mining, is somewhat similar. You hunt through vast sets of data looking for the nuggets which may be useful.
Machine Learning puts Data Mining out of business
There’s a reason that searches of “Machine Learning” are shooting past “Data Mining”, as the results are better and the old “hunt and peck” approach of data mining is being replaced with intelligent agents based on algorithms.
Machine Learning is a species of AI that allows software to re-program itself when exposed to new data, in effect, “learning” without being explicitly programmed. Machine learning is different from Data Mining in which databases are examined by humans. Machine learning is letting the machine decide which data to use, how to get it, and coming up with an answer. Critically Machine Learning has feedback loops so that it gets better over time and when exposed to more data.
Natural language interfaces puts the learning in the hands of the end user data consumer. They don’t need to ask IT to run a report, or try and learn SQL or R.
There’s a reason there are no more coal miners in Lancashire. Newer technologies such as oil, access to the vast coal fields in Canada and mine mechanisation meant that chipping away at a coal seam with a pick axe was no longer the best way to satisfy the expanding British Empire’s energy needs.
It’s probably time to lay down our Data Mining pick axes as well and embrace these new technologies to satisfy our companies endless thirst for information.
For more information on how SAP is using Machine Learning to deliver the Intelligent Enterprise, take a look here.
