Deep Learning and Neural Nets

As I’m typing this, I’m thinking to myself, I hope everyone that reads this knows about the new Nanodegree program I just launched with my partners at Udacity. Do you know about my new program? I hope so, because it just launched. It’s called Siraj Raval’s Deep Learning Nanodegree Foundation program, and I’m hyped about it! We’re going to be taking deep learning to thousands and thousands of new students, and the classroom is about to open (enroll by January 20th at 11:59pm to get in). I just did a dope interview with TechCrunch about the program, and you can read about the launch on the Udacity blog.

Since this is all about establishing deep learning foundations, let’s dive right in.

Deep learning covers a lot of techniques and algorithms, but they all aim for the same basic goal: understanding expansive sets of unlabeled data.

When I say understanding I’m talking about transforming input into useful output, like transforming medical scans into cancer diagnoses, or transforming radar data into steering decisions for a self-driving car. These complex transformations tend to come from a series of simpler transformations, organized into layers. The more transformations, the more complex the relationship between input and output. Deep learning refers to networks of transformations with lots of layers, called a deep network.

Let’s start by discussing neural networks. Neural networks are an approach to artificial intelligence modeled after the human brain. We know the human brain is made of billions of interconnected neurons, and similarly, neural networks are simulated networks of interconnected nodes. The neural network is trained on enormous quantities of data, and each layer of neurons develops a task to perform on the input to move from input to output. These types of neural networks underlie many of the greatest AI advancements of the past few years, from new achievements in accurate medical diagnoses to Google’s triumph with AlphaGo.

Neural networks come in many forms. In our program, we’ll focus largely on three kinds: feed-forward neural networks, convolutional neural networks, and recurrent neural networks.

Feed-forward neural networks most closely match our description above: you have some input, a series of layers of artificial neurons, and ultimately some output. For example, we might turn last quarter’s sales numbers into a projection for the areas to target in the next quarter.

Convolutional neural networks are a class of feed-forward neural networks specifically inspired by the visual cortex, and are particularly well-suited to recognizing visual input. These types of neural networks are powerful when the entire input is available at once, as with an image; think machine learning-generated visual artwork. They can also process medical images or visual road data.

Recurrent neural networks, on the other hand, organize these layers into internal cycles, which allows these networks to process input where time is a factor. This is what allows recurrent neural networks to work in areas like speech recognition and natural language processing: they begin working as soon as input starts coming in rather than waiting for the speaker to finish their sentence.

In our program, we’re going to apply these new deep learning techniques and algorithms — like the types of neural networks mentioned above — to a ton of interesting problems; medical diagnosis, autonomous driving, and game-playing — these are all domains where we can use these powerful techniques.

So, you’re going to learn by building neural networks …. to address IRL problems! No theory just for the sake of theory, we’ll keep it real.

See you in the classroom!