In-Depth on Udacity’s Self-Driving Car Curriculum

Last night we offered acceptances to thousands of students who are excited to join Udacity’s Self-Driving Car Nanodegree Program!

We are working hard to make this the world’s best training program for self-driving car engineers. The entire curriculum will consist of three terms over nine months. Here’s what in the program:

Term 1


  1. Meet the instructors — Sebastian Thrun, Ryan Keenan, and myself. Learn about the systems that comprise a self-driving car, and the structure of the program.
  2. Project: Detect Lane Lines
    Detect highway lane lines from a video stream. Use OpenCV image analysis techniques to identify lines, including Hough transforms and Canny edge detection.

Deep Learning

  1. Machine Learning: Review fundamentals of machine learning, including regression and classification.
  2. Neural Networks: Learn about perceptrons, activation functions, and basic neural networks. Implement your own neural network in Python.
  3. Logistic Classifier: Study how to train a logistic classifier, using machine learning. Implement a logistic classifier in TensorFlow.
  4. Optimization: Investigate techniques for optimizing classifier performance, including validation and test sets, gradient descent, momentum, and learning rates.
  5. Rectified Linear Units: Evaluate activation functions and how they affect performance.
  6. Regularization: Learn techniques, including dropout, to avoid overfitting a network to the training data.
  7. Convolutional Neural Networks: Study the building blocks of convolutional neural networks, including filters, stride, and pooling.
  8. Project: Traffic Sign Classification
    Implement and train a convolutional neural network to classify traffic signs. Use validation sets, pooling, and dropout to choose a network architecture and improve performance.
  9. Keras: Build a multi-layer convolutional network in Keras. Compare the simplicity of Keras to the flexibility of TensorFlow.
  10. Transfer Learning: Finetune pre-trained networks to solve your own problems. Study cannonical networks such as AlexNet, VGG, GoogLeNet, and ResNet.
  11. Project: Behavioral Cloning
    Architect and train a deep neural network to drive a car in a simulator. Collect your own training data and use it to clone your own driving behavior on a test track.

Computer Vision

  1. Cameras: Learn the physics of cameras, and how to calibrate, undistort, and transform image perspectives.
  2. Lane Finding: Study advanced techniques for lane detection with curved roads, adverse weather, and varied lighting.
  3. Project: Advanced Lane Detection
    Detect lane lines in a variety of conditions, including changing road surfaces, curved roads, and variable lighting. Use OpenCV to implement camera calibration and transforms, as well as filters, polynomial fits, and splines.
  4. Support Vector Machines: Implement support vector machines and apply them to image classification.
  5. Decision Trees: Implement decision trees and apply them to image classification.
  6. Histogram of Oriented Gradients: Implement histogram of oriented gradients and apply it to image classification.
  7. Deep Neural Networks: Compare the classification performance of support vector machines, decision trees, histogram of oriented gradients, and deep neural networks.
  8. Vehicle Tracking: Review how to apply image classification techniques to vehicle tracking, along with basic filters to integrate vehicle position over time.
  9. Project: Vehicle Tracking
    Track vehicles in camera images using image classifiers such as SVMs, decision trees, HOG, and DNNs. Apply filters to fuse position data.

Term 2

Sensor Fusion



Term 3

Path Planning



Term 2 and Term 3 are under construction and we’ll share more details on those as we finalize the curriculum and projects.

All of this, including Term 1, is subject to change as we update the curriculum over time, because part of building a great course is taking feedback and making improvements!

If you’ve been accepted into the course, congratulations! We are excited to teach you.

If we suggested you brush up on a few topics and take a self-assessment before joining the course, please do! We are excited to teach you and want to make sure you have a great experience.

And if you haven’t yet applied, please do! We are taking applications for the 2017 cohorts and would love to have you in the class.