Photo by @jessedo81

4 Steps for Learning Deep Learning

A handy list of resources to help you become a deep learning expert

Firstly, if you need some basic information or convincing on why Deep Learning is having a significant impact, check out the following video by Andrew Ng

Andrew Ng: Deep Learning, Self-Taught Learning and Unsupervised Feature Learning

Step 0: Learn Machine Learning Basics

(Optional but recommended)

Start with Andrew Ng’s Class on machine learning His course provides an introduction to various Machine Learning algorithms are out there and, more importantly, the general procedures/methods for machine learning, including data preprocessing, hyper-parameter tuning, etc.

Read the NIPS 2015 Deep Learning Tutorial by Geoff Hinton, Yoshua Bengio, and Yann LeCun for an introduction at a slightly lower level.

Step 1: Dig into Deep Learning

My learning preference is to watch lecture videos and thankfully there are several excellent courses online. Here are few classes I liked

If you are more into books, here are some excellent resources. Go ahead and check them out, I won’t judge.

Step 10: Pick a focus area and go deeper

Identify what you are passionate about and go deeper. The field is vast, so this list is in no way a comprehensive list.

1. Computer vision

Deep learning has transformed this area. Stanford’s CS231N course by Andrej Karpathy’s course is the best course I have come across; CS231n Convolutional Neural Networks for Visual Recognition. It teaches you the basics and up to covnets, as well as helping you to set up GPU instance in AWS. Also, check out Getting Started in Computer Vision by Mostafa S. Ibrahim

2. Natural Language Processing (NLP)

Used for machine translation, question and answering, sentiment analysis. To master this field, an in-depth understanding of both algorithms and the underlying computational properties of natural languages is needed. CS 224N / Ling 284 by Christopher Manning is a great course to get started. CS224d: Deep Learning for Natural Language Processing, another Stanford class by David Socher (founder of MetaMind)is also an excellent course which goes over all the latest Deep learning research related to NLP. For more details see How do I learn Natural Language Processing?

3. Memory Network (RNN-LSTM)

Recent work in combining attention mechanism in LSTM Recurrent Neural networks with external writable memory has meant some interesting work in building systems that can understand, store and retrieve information in a question & answering style. This research area got its start in Dr. Yann Lecun’s Facebook AI lab at NYU. The original paper is on arxiv: Memory Networks. There’re many research variants, datasets, benchmarks, etc that have stemmed from this work, for example, Metamind’s Dynamic Memory Networks for Natural Language Processing

4. Deep Reinforcement Learning

Made famous by AlphaGo, the Go-playing system that recently defeated the strongest Go players in history. David Silver’s (Google Deepmind) Video Lectures on RL and Professor Rich Stutton’s Book is a great place to start. For a gentle introduction to LSTM see Christopher’s post on Understanding LSTM networks & Andrej Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networks

5. Generative Models

While discriminatory models try to detect, identify and separate things, they end up looking for features which differentiate and do not understand data at a fundamental level. Apart from the short-term applications, generative models provide the potential to automatically learn natural features; categories or dimensions or something else entirely. Out of the three commonly used generative models — Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs) and Autoregressive models (such as PixelRNN) , GAN’s are most popular. To dig deeper read

Step 11: Build Something

Doing is key to becoming an expert. Try to build something which interests you and matches your skill level. Here are a few suggestions to get you thinking:

For more inspiration, take a look at CS231n Winter 2016 & Winter 2015 projects. Also keep an eye on the Kaggle and HackerRank competitions for fun stuff and the opportunities to compete and learn.

Additional Resources

Here are some pointers to help you with continuous learning

See ChristosChristofidis/awesome-deep-learning, a curated list of awesome Deep Learning tutorials, projects and communities for more fun