Alex
Alex
Jul 25, 2017 · 2 min read

I don’t think this is a correct ‘break-down’ of what “conventional statistics” (I would call ‘traditional statistical analysis’) is and what “machine learning” is.

The majority of machine learning is implementing well known (often pretty old) statistical methods: regression, k-means clustering, Gaussian variational methods etc. And prediction is certainly not the soul-dominion of ML! Thomas Bayes’s equation was revealed in the late 1700’s and has been used ever since in “predicting” latent variables.

Much of machine learning is just age-old statistical fitting: fitting some chosen statistical model to data. The difference is, whereas in the old days we’d only be interested in the “goodness of fit” or prediction performance on some test set, nowadays we’re also worried about the computational complexity of the fitting algorithm — how long does it take to converge? How can we speed it up? What are the costs?

A lot of ML fits into this category of “computing statistical fits” (k-means, PCA, ICA…) — but! Admittedly there is a lot of “new stuff” that has been developed (or at least much advanced) under the label “machine learning”. Examples are as support vector machines (which, btw, emerged from the field of “statistical learning theory”), kernel methods, and neural network methods. In some of these techniques, the “statistics” (or “features”) are hidden from the user, so perhaps they don’t seem like statistical methods. So perhaps some of these sit better under the ML name.

In general, one of the things I hate about the ML industry (which I think it inherited from Comp Sci) is this affinity to constantly make up new, “sexy”, hardly-ever-intuitive names for everything! Especially when many of the approaches existed in physics / statistics / mathematics already!

OK, closing rant over. I hope this in interesting. :-)

Alex

Written by

Alex

Applied mathematician. Working on Learning — in machines and biology. Interests: conservation & tech. Gratefully listening to Bruch, Franklin, Kaytranada