How come deep learning took so much time to take off

Photo by Markus Spiske on Unsplash

Maybe the worse thing that can happen to an idea is being born on the wrong moment, or/and even wrong place.

Photo by Michal Matlon on Unsplash

Take the case of YouTube, was is the first video streaming platform? not, not at all! But it was born on the right moment!

“In 1999–2000 it was too hard to watch online content you had to put codecs in your browser and do all this stuff [about company that failed two years before YouTube]” Bill Gross

It was somehow similar with deep learning, since the act adding more hidden-layers is not new, and it is even straightfoward: anyone with a outside thinking could try that out, and have succeeded if we had the proper tools.

What made deep learning just now?

Photo by Aron Visuals on Unsplash

Here goes three reasons:

 — Hardware

 — Datasets and benchmarks

 — Algorithmic advances

Hardware

Indeed, it is amazing how fast hardware evolved, in special for personal usage. I still remember the power of my first computer, and second, and the difference. Nonetheless, GPUs (Graphical Processing Units)were the most important step forward. What surprise me is that they were developed for gaming, and becoming so important to scientific computation.

Photo by Fredrick Tendong on Unsplash

Datasets and benchmarks

It still surprises mehow much we can find nowadays online for free, datasets: in the close past, datasets were precious stones, people kept them in banks; in Brazil, we still do that!. The concept of Big Data appeared as an output from the huge amount of data, where the attention shifted from having the dataset to actually making sense of it. Several groups make the dataset online, for free, since they cannot alone make sense of the huge amount of data they generated.

Photo by Markus Spiske on Unsplash

Algorithmic advances

Even though it seems we did not evolve too much on the basics, as Marvin brought to attention, we actually have better versions of old routines, and better software to actually running them! What I like to say, AI are old algorithms dressed with new computer power.

Photo by Андрей Сизов on Unsplash

Final remark

I came accross Network networks about in 2008–2009, when a professor showed me a simple perceptron, on a course, it was a simple classification problem. At the time, it was still hard to solve problems with complex nature, and most of the problem we can solve today were a dream. I hope we keep evolving! At the time, AI was a tool, not it is THE tool.

Main References

Deep Learning with JavaScript: Neural Networks in TensorFlow.js. Book by Eric D. Nielsen, Shanqing Cai, and Stanley Bileschi. Section: 1.1.4 Why deep learning? Why now?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store