May 21
How come deep learning took so much time to take off
Maybe the worse thing that can happen to an idea is being born on the wrong moment, or/and even wrong place.
Take the case of YouTube, was is the first video streaming platform? not, not at all! But it was born on the right moment!
“In 1999–2000 it was too hard to watch online content you had to put codecs in your browser and do all this stuff [about company that failed two years before YouTube]” Bill Gross
It was somehow similar with deep learning, since the act adding more hidden-layers is not new, and it is even straightfoward: anyone with a outside thinking could try that out, and have succeeded if we had the proper tools.
What made deep learning just now?
Here goes three reasons:
— Hardware
— Datasets and benchmarks
— Algorithmic advances
Hardware
Indeed, it is amazing how fast hardware evolved, in special for personal usage. I still remember the power of my first computer, and second, and the difference. Nonetheless, GPUs (Graphical Processing Units)were the most important step forward. What surprise me is that they were developed for gaming, and becoming so important to scientific computation.
Datasets and benchmarks
It still surprises mehow much we can find nowadays online for free, datasets: in the close past, datasets were precious stones, people kept them in banks; in Brazil, we still do that!. The concept of Big Data appeared as an output from the huge amount of data, where the attention shifted from having the dataset to actually making sense of it. Several groups make the dataset online, for free, since they cannot alone make sense of the huge amount of data they generated.
Algorithmic advances
Even though it seems we did not evolve too much on the basics, as Marvin brought to attention, we actually have better versions of old routines, and better software to actually running them! What I like to say, AI are old algorithms dressed with new computer power.
Final remark
I came accross Network networks about in 2008–2009, when a professor showed me a simple perceptron, on a course, it was a simple classification problem. At the time, it was still hard to solve problems with complex nature, and most of the problem we can solve today were a dream. I hope we keep evolving! At the time, AI was a tool, not it is THE tool.
Main References
Deep Learning with JavaScript: Neural Networks in TensorFlow.js. Book by Eric D. Nielsen, Shanqing Cai, and Stanley Bileschi. Section: 1.1.4 Why deep learning? Why now?
