Member-only story
What Occam’s Razor Means in Machine Learning
A practical explanation of this law of simplicity
“Everything should be made as simple as possible, but not simpler.”
— Albert Einstein
A while ago, I wrote an article about the No Free Lunch Theorem — a theorem inspired by philosophy that implies that no single machine learning algorithm is the universally best-performing algorithm for all problems.
Like the No Free Lunch Theorem, another concept in machine learning that has origins in philosophy is Occam’s Razor. Occam’s Razor basically states that given competing theories and explanations, the simplest ones should be preferred. In the words of Ockham, “entities must not be multiplied beyond necessity.”
Philosophical Origins and Formulations
Occam’s Razor is an idea that may have originated from earlier philosophers. In fact, Aristotle (384–322 BC) stated that “we may assume the superiority, other things being…
