The Deep History of Deep Learning… Part II
Breakfast Clubs to Board Games
In Part I, we left our story with a double side of Bacon. So why not begin Part II with a little more breakfast. Returning to England, Cambridge this time, we encounter the world’s second most famous Breakfast Club. They met more than once, seem to be all Brains, and changed the world as we know it.
Let’s start with the ringleader. William Whewell took the teachings of the Bacons and created the “scientist”. Truly, he coined the word. He also created the Philosophical Breakfast Club, from its four core members and various associates — the world of modern technology and science exploded. Thus simultaneously proving that all great things start with Bacon AND breakfast IS the most important meal of the day.
Our club members include:
William Whewell — the first “scientist” & “physicist”
John Herschel — who gave us photography
Charles Babbage — who gave us the computer
And Richard Jones — who may have inspired the “dismal” part of the “dismal science”
These four intrepid souls would form the nucleus of an even larger network of newly minted (or at least named) “scientists”. Associates like Charles Darwin (evolution), George Peacock (algebra), Ada Lovelace (computer programming), and Charles Lyell (geology) represent only a handful of a massive whose who.
This network would deliver on both the further development of Aristotle’s core principles as well as their application. They would also successfully merge these endeavors with a new appreciation for experimentation, statistics, and quantification.
Every failure is a step to success. — William Whewell
Let’s Take A Deep Breath
Since we began our story, we have traveled thousands of miles over thousands of years. We have followed the steps of Aristotle from the Lyceum of Greece to the Halls of Cambridge. We have witnessed the evolution of science, the birth of evolution, and the dawn of the computer (analytic engine). Along the way, we touched quite a few world religions, enjoyed two sides of Bacon, and joined a Breakfast Club.
This should call for some 80’s style celebration. Only our journey has only just begun. While the first two millennia were quite a ride — a bit like hitchhiking through the universe, the next two centuries have more to tell. So where to next?
Across The Pond, Of Course…
Our next step in this journey brings us to Cambridge… uh, Massachusetts this time. Only briefly as our wandering band of scientist-philosophers will soon bounce to Georgetown, MIT, and New Jersey (with a small dose of Stanford for fun and better weather).
At this point, let me interject an interesting aside. While nearly all science has been carried along on this journey, perhaps none more so then the field of optics. It’s history is written in nearly all these same men (sorry, Lovelace was not part of that story). And here it arrives again…
Charles Sumner Tainter began his career in Cambridge (MA) making telescopes for the US Navy. He soon came in contact with Alexander Graham Bell & his chemist cousin Chichester. Together Charles & Chichester would start Bell Labs in Georgetown, which soon became Volta Laboratory (before going back to Bell Labs and moving to New Jersey). Volta Labs was the birth place of telecommunications, fiber optics, and the American leg of our journey. It also introduced the instrumentation side of modern analytics (but that is an article in its own right).
From here our story takes a series of steps through the first half of the twentieth century. It includes names like E.H. Moore, Ana Wheeler (creator of linear algebra), Oswald Veblen, Alonzo Church, and Alan Turing. Church and Turing would pick-up where Babbage and Lovelace had left off — propelling computer science to new levels. It is an iterative tale which is quite fitting for this story but alas an article for another day.
What is most important here- Alan Turing liked chess… well actually, probably not. It seems he wasn’t very good at it. He was however hell-bent on proving a computer could be programmed to play it. He succeeded in creating a supervised set of directions that he could enter into a computer to calculate chess moves.
What is more important? Where are story goes next.
More Board Games!
Enter Arthur Samuel, creator of “machine learning” and one heck of checkers player. Not him per se, but the computer he “taught” to play it. Samuel was a student at MIT, a study of Turing, and a member of Bell Labs (now in NJ). He is also the only guy to really make checkers interesting.
Now it is important to note that once again, Samuel coined the phrase “machine learning”. He didn’t invent it anymore than Whewell invented “scientists” or Darwin invented “evolution”. As is often the case, history is better at recording and rewarding those who popularize terms. Most inventions require a series of steps and actors, as illustrated by our current story which has finally arrived!
Because, what Samuel did create, more so than popularize, was Deep Learning. Now you won’t see this directly stated anywhere but here (at least as of this writing). I think this is partially because Deep Learning is still being popularized, partially because we are talking checkers, and as is often the case because he didn’t coin the term.
In 1959, Arthur Samuel defined machine learning as a “Field of study that gives computers the ability to learn without being explicitly programmed”.
Regardless, Samuel applied Deep Learning! He “taught” a computer program through layered algorithms and feedback to play the game of checkers better than he himself was able to. It even beat a self-proclaimed expert a few years later at the IBM labs.
If you are looking to review the code — or at least a close approximation — try this link:
There is still more to this tale…
We began with a breakfast club in England and moved to the laboratories of the United States. We covered nearly a century of “scientists” and a decade of “machine learning”. And yet, we are still six years away from the documented first appearance of Deep Learning. In our next article, we will step into the 1960’s and final half century of this Deep History. Thanks for reading!
For more on the Breakfast Club consider:
