Udacity Self-Driving Car Engineer Nanodegree: A walkthrough

Dominic Monn
Aug 26, 2017 · 6 min read
Udacity’s “Clara” — Their very own Self-Driving Car. Image: Udacity and David Silver

About a year ago, the well-known online education provider Udacity announced their newest program: The Self-Driving Car Engineer Nanodegree. I had known Udacity for quite some while at that point. I had done a few free courses before and was actually thinking about starting one of their Nanodegrees. But when they announced the Self-Driving Car Engineer Nanodegree (or SDCND for short), I was immediately hooked.

At that point, I was in the fourth year of a 4-year apprenticeship in Software Engineering (to read more about the apprenticeship model in Switzerland click here) and was looking for a new challenge in the Machine Learning space. So I was giving it a shot.

The SDCND was a bit different from the previous Nanodegrees: People had to apply to grab one of the first 250 spots in the program, there were hiring partners such as Mercedes-Benz, Uber ATG, BMW and NVIDIA in place, it consisted of three fixed three-month long terms and there was a mentorship program. I applied for the first cohort and about a month after I got a mail. I was accepted into the second (November 2016) cohort! Now, almost a year after that, I’m only days away from completing the program. Therefore, I want to share my experiences.

Term 1: Deep Learning and Computer Vision

Term 1 consisted of five projects and various lessons centred around Deep Learning and Computer Vision. To be fully honest, this was the term I enjoyed the most since I wanted to learn as much as I could about Deep Learning.

The first project was an easy one: Using some basic Computer Vision techniques, we annotated lane lines in a video. I remember that I was excited submitting my very first project and I got an awesome review (from an actual human!). I was also excited about the community forming in their Slack channel. I actually made some friends there and had some really good conversations.

The second project was to build a Traffic Sign Classifier. In hindsight, this is a very basic and easy project. But as a beginner, to build this was a huge challenge. However, after some days I got it running and it actually got a great review.

The third project might be my all-time favourite project from the program. Using Udacity’s own race track simulator, we built a behaviour cloning network. First, data had to be collected by driving around the track a few times (P.S: Use a controller for this one!). Then, a neural network had to be built to predict steering angles. The result was a “real” virtual autonomous car. I was so proud!

The fourth project was similar to the first one: Advanced Lane Finding. This time, we used some more advanced computer vision techniques to get better results (e.g. on roads with shadows and curves).

The fifth and final project was a tough one! Using computer vision and machine learning techniques, we built a vehicle tracking algorithm. It was not easy to train the machine learning algorithm to correctly classify cars. This was a lot of work, but with this one done, the first term was completed.

To my surprise, I had finished the term a lot quicker than expected. I actually could start the second term with the first cohort. This was exciting since I’d be one of the first people to experience the content, but it was also scary, since there weren’t as many people to help me. At this point, I also got the offer to mentor some students myself. I did not use my mentor very much, but I took the offer and started with 30 students. At the time of writing, I’m mentoring over 80 students!

Term 2: Sensor Fusion, Localization and Control

The second term was the least exciting one for me personally. I suppose people who love robotics will absolutely love this term, but I did not have any background in C++ and wanted to do more Deep Learning. I was hungry for more!

This term consisted of five projects as well. With help of Mercedes-Benz, we learned a lot about Extended and Unscented Kalman Filters, which we both had to implement for projects.

The third project was about Kidnapped Vehicles. Using a particle filter and a map, we had to localize an autonomous vehicle in an unknown environment. This one was very fun to do!

There wasn’t much theory in this term (what a lot of my peer students did not like), so after a single theory block, the next project already started. The goal of this one was to build a PID controller (proportional–integral–derivative). We also used Udacity’s simulator for this one.

Final project (MPC). By Udacity

The final project of the second term was to build a model predictive control. This was a really interesting project and gave a lot of insight into an autonomous vehicle. The goal of the project was to steer the car around a track with latency between the commands. Very exciting!

At this point, I was confident enough to go on Deep Learning interviews. My apprenticeship was coming to an end and I was looking for something new. I failed at some of the technical interviews which, even if it hurt my confidence a bit, taught me a lot. My advice for people looking for a job in a new industry: Go on interviews. It’ll show you all your flaws and what to study next. In the end, I got an amazing internship at NVIDIA in Zurich.

Term 3: Path Planning, Concentrations and System Integration

The final term was another exciting one, even though it only has three (or four) projects.

In the path planning project, the goal is to build a path planner to drive on a high way without so-called “incidents”. Which means that the car shouldn’t drive faster than the limit — and also not a lot slower — and of course it should not crash into other cars. This project was a lot of fun!

In the next part, students have the possibility to choose between to concentrations: Advanced Deep Learning and Functional Safety. Of course, I choose Advanced Deep Learning which was taught by the Deep Learning Institute of my future employer. In this concentration, students learn about Fully Convolutional Networks and will build a Semantic Segmentation algorithm to identify free areas on the road.

The capstone project, however, is the truly special thing about the SDCND. Students are required to build teams of 3 to 5 people and work on the last project together. The goal of this one is to implement some nodes in the Robot Operating System (ROS), test the solution in a simulator and then load it onto the Udacity Self-Driving Car “Clara”! I had a look at the team spreadsheet and got into a team named “WhiteDriver” which consisted of four people already, which were mostly in my timezone. On the same evening, we planned the project.

The “WhiteDriver” issue board

Our team leader implemented some things very fast and other people from the team did too. Unfortunately, my computer which I use for working on the nanodegree broke in this very week, so I was left with a weaker machine which could not run the simulator. Therefore, I implemented the computer vision algorithm to classify traffic lights. This did not need me to run the simulator or a virtual machine.

At the time of writing, we are finishing this project and are expecting to submit it somewhen next week. After that, our journey in the SDCND is over! I’m thanking David Silver, Udacity and the whole SDCND team for putting this program together. I have learned a lot and it helped me to transition into a new career. I will never forget that.

If you want to enter the Self-Driving Car Engineer Nanodegree yourself, do so by clicking the link and applying to the program. The views expressed in this article are my own!

)

Dominic Monn

Written by

Deep Learning Engineer & Maker.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade