What’s the difference between deep learning and regular statistics?

Statistics is about fitting a simple model to noisy data, where the central problem is to avoid being misled by noise.

Deep learning is about trying to fit a complex model to highly structured data, where the main problem is figuring out how to learn and represent that structure. Although both techniques are applied to large datasets, the process is quite different.

What do we mean by “structure”? And how do we go about fitting such a thing to out input data? What should the input data even be? What can we use these structural representations of data for?

These are all complex questions. The best way to answer it is that there are a number of neural network designs that have been found to be very good at representing and processing data.

As one HN comment put it:

The problem is not that people don’t know how to build neural networks.
The problem is that people don’t understand what they can do with this technology, or even, in a meaningful sense, what it is.
Given a array of input numbers [i1, i2, i3…] it provides a black box that can map that input to a set of output numbers [o0, o2, …] where the dimension of the input and output are different.
That’s. All. It. Does.

Neural networks are kind of like a mysterious black box magic function whose uses only become apparent after wading through hundreds of academic papers.

Questions like these are what motivated me to create Neural Networks for Hackers, a MOOC focused on the practical side of deep learning.

The course covers:

  • what software packages and frameworks to use
  • training models using GPUs, setting up a cluster in the cloud
  • a broad overview of algorithms used in neural networks — in plain English
  • understanding the vocabulary used in deep learning academic papers
  • how to keep up with the deep learning literature
  • interesting project ideas so you can start building right away

The crowdfunding project is below. All the course material is already written, the fundraising is purely for creating more diagrams, audio recordings, screencasts and more.