Deep Learning MOOCs
With deep learning becoming more accessible to developers, new courses and programs are being launched at an unprecedented pace. There are abundant resources for studying deep learning.
I get this question a lot: “Which online deep learning program should I choose?” My answer is: “ It depends.” There are so many choices nowadays, which one to choose depends on your learning style, time or money you can afford, and your career goals. I’m going to share my perspective on a few of the well-known deep learning MOOCs:
- Udacity — Deep Learning Nano degree.
- Coursera — Deep Learning Specialization.
- Stanford —CS20 (TensorFlow for Deep Learning Research), CS230 (Deep Learning), CS231n (CNN for Visual Recognition) & CS224n (NLP with Deep Learning).
- MIT — Introduction to Deep Learning.
- Fast.ai — Deep Learning for Coders.
- Google — Machine Learning Crash Course with TensorFlow APIs.
Udacity Deep Learning Nano degree
Link to program | Cost: $999 or $84 per month
Pace: assigned to a cohort with other students. This program is fairly challenging so be prepared to spend at least 10 to 15 hours per week. You are expected to complete the program within 4 months with 1 month extension. If you don’t finish by the deadline then you will be dropped out. Then you will need to pay the full price again to re-enroll.
Where to go for help?
- Slack group and channels for each topic. Office hours available as well.
- Mentor — each student used to get assigned a mentor. Now instead you use study group.
- Study Group — a chat system inside of the Udacity portal. replaced the mentor system.
- Discussion forums—where students can help each other out with their questions.
Course Structure: the nanodegree is organized into 6 parts, each with a few lessons:
- Intro to Deep Learning
- Neural Networks
- Convolutional Networks
- Recurrent Networks
- Generative Adversarial Networks
- Deep Reinforcement Learning
Lectures were taught with both TensorFlow and Keras. There are several projects written in low level TensorFlow. Students are not expected to do the data pre-recessing and will focus on model architecture. Students used to need to create conda environments themselves and train on GPU on either AWS or FloydHub. Now the nanodegree provides environments with enough free GPU hours to complete the projects.
Students who graduated will have guaranteed admission into the Self-Driving Car and Flying Car nanodegree programs.
Other notes: The Udacity lectures are taught by several top notch instructors with quality content. The nano degree is full of cutting edge information in deep learning. It also offers career assistance. The deep learning is one of several nanodegrees from Udacity School of AI. You can choose to continue with another nanodegree such as Artificial Intelligence, Robotics, or Reinforcement Learning after finishing this one.
Coursera Deep Learning Specialization
Link to program | Cost: $49/month
Pace: courses are organized by weeks so you don’t need to spend extra time planning. If you fall behind, just switch to the next month co-hort. It’s a very flexible program and fairly affordable.
Where to go for help?
Discussion forums are the only way to find help if you get stuck. This program is very well structured. Many challenges you encountered are perhaps tools issues or some concepts which you can get answers from the forum discussions.
Program structure
There are 5 courses in this specialization:
- Intro to neural networks
- Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
- How to structure your deep learning project
- Convolutional Neural Networks (CNN)
- Sequence Models
Each course is organized by weeks, and each week has video lectures with transcripts, followed by a quiz then one or more programming exercises. Each programming exercise is well written with detailed guidance on what the student is supposed to do. There are also guest speakers for each week so you get perspective from the industry.
Other notes: Dr. Andrew Ng is an excellent professor and he is great at explaining complex concept in a way that is easy to understand. The program provides a solid foundation to deep learning.
Stanford Deep Learning Coureses
There are several courses from Stanford on deep learning so I will not cover each of them in details. Rather I’m listing out the ones that I know of:
- CS20 (TensorFlow for Deep Learning Research) — great for learning TensorFlow, as well as an intro to deep learning. You will learn topics such as TensorFlow, Keras, CNN, RNN, GANs and Reinforcement learning.
- CS230 (Deep Learning) — this course is pretty much the same as the Coursera one above, but without the videos.
- CS231n (CNN for Visual Recognition)- great introduction to computer vision with deep learning.
- CS224n (NLP with Deep Learning)- focuses on natural language processing with deep learning.
All of the slides and notes are free and accessibly to the public with videos from previous years available.
MIT Intro to Deep Learning
Link to program | Cost: free
Pace: self-paced.
Program structure:
There are five sessions with two videos per session:
- Two videos: Intro to Deep Learning & Deep Sequence Modeling. Lab: Intro to TensorFlow + Music Generation with RNNs.
- Two videos: Deep Computer Vision & Deep Generative Models. Lab: Disease Detection from Human X-Ray Scans.
- Two videos: Deep Reinforcement Learning & Limitations and New Frontiers.
- Guest lectures from Google and NVIDIA.
- Guest lectures from IBM and Tencent.
The lectures are well organized and in top quality. This is a good fit if you would like to go through various deep learning topics in a short period of time. I also find guest lectures from industry very interesting.
Fast.ai Deep Learning for Coders
Link to program | Cost: free
Pace: self paced. Be prepared to spend 1 to 3 weeks per lesson depending on how comfortable you are with the topics covered in the lesson, assuming you study 10 to 15 hours per week.
Where to go for help?
- Wiki page — one of the things that I really like about fast.ai are the wiki pages for each lesson. I included link to lesson 1 wiki as an example. As you see there are tons of learning resources with links to articles, research papers and student notes which are amazing and can help you to better understand the videos.
- Discussion forum — head over to http://forums.fast.ai/ if you get stuck or have questions. The forum has students worldwide actively participating.
Course Structure:
The program is divided into two parts with 7 lessons in each. Each lesson consists of a 2-hour video and a wiki page. In each video Jeremy goes over code in Jupyter Notebooks and deep learning concepts.
Since the lectures were recorded for in-class lessons then later on converted to online MOOC, the Jupyter notebook numbers may not match up with the lesson number. Also depending on your background knowledge, you may need supplement learning materials in order to fully understand the videos.
Other notes:
Fast.ai is unique and unconventional. It’s very different from the other programs in these ways:
- It adopts a teaching style of top-down approach focusing on hands-on experience instead of theories.
- The lectures jump back and forth a bit between CNN and RNN, unlike the other programs where the students learn CNN and then RNN.
- Sample code uses its own fast.ai library written on top of PyTorch.
- It has its own blog post page with great information.
- You may need to figure out your environment setup for training on GPU which resembles the challenges you may face in the real world. (Note: I was able to run the Part 1 notebooks in Colab with additional code for installing packages & downloading datasets).
- The program focuses on optimization techniques so the accuracy of the models are much higher than what you will see from the other MOOCs.
- You will find lots of fast.ai students on Kaggle, showcasing the state of art techniques they learned from the program.
Protip: if you lack the basics on ML or deep learning, you may find this curriculum a bit challenging to keep up with. Fast.ai does have intro to ML videos. Also since this MOOC is so different from the others, you may benefit from taking another deep learning MOOC before or after completing the fast.ai one.
Google ML Crash Course with TensorFlow API
Link to program | Cost: free
Pace: self paced.
Where to go for help?
On meetup.com search for a Google Developer Group (GDG) near you and see if they are running a Study Jam. You can join the Study Jam to attend in person sessions and to study with others online together.
Other notes:
This program was used internally to train thousands of Google engineers and then open sourced early 2018 to the public. All concepts are well explained with visualization to help students better understand. For example, take a look at the explanation of what is learning rate and the exercise on how to optimize learning rate.
It’s excellent at teaching the machine learning basics before moving on to neural networks and deep learning. It also has a very helpful pre-requisites page and a ML glossy page.
Protip: this MOOC uses low level TensorFlow which is great for studying fundamentals. You should also check out the Keras guide and tutorials on TensorFlow.org to get familiar with the modern TensorFlow high level APIs. All the tutorials can be opened and run in Google Colab with free GPU.
There are many other excellent online learning resources such as Siraji Raval’s School or AI, or Kaggle’s learn.
