The Deep Learning Playbook
Machine learning is the art and science of making predictions based on previous observations. Deep learning without doubt is THE hottest subfield of machine learning and it is ubiquitous in various state-of-the-art methods for problems like speech recognition (audio), object recognition (vision), natural language processing (communication). People are studying this technique and try to reveal its potential in all kinds of applications from finance to medicine. I believe you can only understand a learning algorithm and know how to use it by hand coding the program and running it on real data. I also believe an easy-to-follow learning path is necessary to keep you motivated and continue. To help those who are interested to know and use deep learning in their problems, I curated a minimal list of resources based on my own learning experience to get your hands dirty on deep learning quickly without being distracted by the flood of materials online. I wouldn’t be surprised if you find some other lists (such as this) that contain much more resources than this one, but again, my goal is to help you get started in a more focused way.
It’s recommended that you follow the materials in order: 1) gain basic understanding of deep learning (model structure, objective, optimization); 2) pick up a tool/framework and familiarize yourself; 3) dive into real problems by exploring and playing with open source projects.
Theory
- [Book] Yoshua Bengio, Ian Goodfellow, Aaron Courville. Deep Learning. 2015.
- [Book] Michael Nielsen. Neural Networks and Deep Learning. 2015.
- [Course] Convolutional Neural Networks for Visual Recognition. 2017.
- [Course] Deep Learning for Natural Language Processing. 2017.
- [Course] Practical Deep Learning for Coders. 2017
- [Course] DeepLearning.ai. 2017
- [Course] TensorFlow for Deep Learning Research
Libraries
- Theano (Python)
- Libraries based on Theano: Lasagne, Keras, Pylearn2
- Caffe (C++, with Python wrapper)
- TensorFlow (Python, C++)
- Torch (Lua)
- ConvNetJS (Javascript)
- Deeplearning4j (Java)
- MatConvNet (Matlab)
Projects / Demos
- All the tutorials of your favorite library above
- Facial Keypoint Detection
- Deep Dream
- Eyescream
- Deep Q-network (Atari game player)
- Caffe to Theano Model Conversion (use Caffe pretrained model in Lasagne)
- R-CNN
- Fast R-CNN
- Plankton Classification (winning solution of National Data Science Bowl on Kaggle)
- Galaxy Classification (winning solution of Kaggle competition)
- University of Toronto Demos
- more to be added…
