How the Major Technology Trends are Driving Deep Learning?

4 min readJan 28, 2018

Science fictions like Star Trek decades ago predicted our future will be surrounded by artificial intelligence, such as smart devices and androids. In reality, artificial intelligence (deep learning/neural network) only became viral and explosive in the recent years. Why? There are a few reasons. Artificial neural network is not a new idea. In fact, it was invented back in 1943, by Warren McCulloch and Walter Pitts [1]. Their research paved the way for later research to split into biological processes and application of neural net to AI. An artificial neural network is basically mimicking how neurons in our brains work, in a much smaller scale.

Big Data

The reason it did not take off was computational power, and lack of training data. Since the internet and social media became mainstream, people around the world have uploaded data (photos, chats, etc) at a tremendous speed. The chart below shows the number of photos uploaded and shared on select popular social media from 2005 to 2014 [2]. In 2014, there were over 1.8 billion photos uploaded and shared.

All the data becomes ideal for large neural network training because an interesting characteristic of a neural network is generally the more data it is fed, the better it gets, similar to how we learn.

Cloud, Cloud, Cloud

Another technology that helps with accessing the data is cloud storage. Following the Moore’s Law, number of transistors double every two years on the same size die [3]. Also the price of data storage is always dropping per MB. If anyone had Hotmail accounts 20 years ago, they would remember they only had 2MB of data to store all their emails. When Google joined the email business a decade ago, they offered a few GB of storage for emails. Nowadays, they are over 100 GB or unlimited. Major tech companies keep building new data centers to store all the data their consumers upload and share. The data is essentially gold to large neural network training. There was hardly much cloud storage back when ARPANET was launched. About half century later, there is over 1 Exabyte of data stored in the cloud, which is equivalent to 1,048,576 Terabytes of data [6]. Without these large cloud storages, it is impossible for neural network to succeed.

High Speed Internet

To facilitate big data transfer for neural network training, high speed internet plays a big role too. Before the internet, as we know it, was invented, there was ARPANET in 1969, with connection speed of 50Kbps. Artificial neural network was already invented by that time, but the transfer speed was nowhere near sufficient to transfer gigabytes of data. Then in 1995, Canada got the first high speed internet service in North America with speed up to 100 times faster than dial-up modem. In 2013, Google Fiber introduced its 1 Gbps internet to US consumers [4]. The plot below shows the progression of average internet speed in the US from 2007 to 2017. It increased 5 time over the last decade [5].

Powerful Computational GPU

For computational power, imagine a large neural network may have 100,000 neurons to learn, and it may take 1000 iterations to find global optimum solution, and there are 10 million training data to use. It easily will take more than 1 quadrillion (1015) calculations to complete. Thanks to incredible breakthrough and achievement in GPU technology in recent decade (figure shown below), it takes a lot less time to train a large neural network compared to 20 years ago. The reason GPUs are perfect candidates for neural network training is because the mathematics in a neural network can be run in parallel and the GPUs are very efficient at calculating thousands of these equations in parallel.

Infinite future

With all technologies booming, what can we expect the superpower — deep learning is going to change our next generations? The portable devices can translate what we say in different languages, and then spoke them in any languages using a simulation of our own voice, even with emotion expression. Driverless car calculates the best way safely and properly take us to wherever the places we want to go. The medical condition is recorded and synchronous with healthy system, so the Dr.IMAGINARY identifies all potential diseases before we see the Dr.REAL.

What else? Welcome to unlimited imagination of the future in deep learning.