Core Technologies Used In Self Driving Cars
By- Vatsal Srivastava
Autonomous systems have suddenly caught the attention of the world with the path-breaking developments in the fields of Computer Vision, Graphical Processing Units (GPUs) and Sensor Technologies.
An autonomous system makes use of many technologies in order to perform its functions in an efficient and safe manner. Let us take the example of a self-driving car. A self-driving car such as Uber’s Self-Driving Car[Image 1], uses a set of cameras for the purposes of identifying various vehicles, pedestrians, objects, traffic lights and lanes. It uses the recent advances in computer vision and deep neural networks, such as, Convolutional Neural Networks (CNNs) and Fully-Convolutional Networks (FCNs) to do this.
Udacity’s Self Driving Car Engineer Nanodegree has many such assignments where one has to implement these functionalities. The images 1, 2 and 3 show some of the functionalities that I implemented during the course of the Nanodegree.
Using the concepts of Transfer Learning, a pre-trained model can be modified for the purposes of detection of different kinds of objects and their classification. This functionality is very important in the real-world autonomous navigation by a vehicle.
Deep learning using Convolutional Neural Networks (CNNs) is being used to detect and classify the traffic lights which can convey the important navigation information to an autonomous vehicle.
One more area where deep learning is being used in autonomous vehicles is the identification of the lanes at pixel level using Fully Convolutional Networks (FCNs). This helps in making sure that all the lane and traffic rules are followed by an autonomous vehicle.
However, an image of the surroundings is not enough to drive safely around the streets of a city. A GPS system is also present to help the car position and navigate itself. But the accuracy of commonly available GPS are about 4 meter RMS[1]. Therefore, in order to improve the accuracy of navigation, it also uses a set of sensors such as Light Detection and Ranging, commonly referred to as LIDAR. A LIDAR works by measuring distance to a target by illuminating the target with pulsed laser light and measuring the reflected pulses with a sensor. Differences in laser return times and wavelengths are then used to make digital 3-D representations of the target. A LIDAR can give an accuracy of upto 2.5 cm[2]. Multiple LIDAR modules throughout the body of the car help in creating an accurate map of the entire surroundings and avoiding blind spots. LIDAR and RADAR play an important role in collision avoidance as well. A sample image[Image 2] of a LIDAR point-cloud output can be seen below.
Multiple sensors of a car are integrated together by a technique called sensor-fusion to help in the accurate functioning of the vehicle. This might sound very complicated in theory, but is actually quite simple once you grasp the fundamentals. I used these concepts of sensor fusion and path planning to design my own self-driving car using a simulator provided by Udacity. In the video below you can see the core technologies in action that are helping an autonomous vehicle navigate a busy highway.
Udacity’s Self Driving Car Nanodegree not only covers the various technologies used, but in collaboration with companies like Nvidia, AWS and Mercedes-Benz also provides hands-on assignments where one gets to code these basic building blocks of such autonomous vehicles. A review of the Self-Driving Car Nanodegree can be found here. There is also an exciting new Flying-Car Nanodegree that teaches about drone robotics, how to develop sophisticated flying car systems, and write real code for real aircraft. Besides these, there are an amazing number of Udacity courses to help you get started in this exciting field of autonomous systems which can be found here.
I hope that you make use of this amazing opportunity by Udacity to gain advanced knowledge in this niche field and help create the next generation of intelligent and safe autonomous systems.
References:
[1] https://www.gps.gov/systems/gps/performance/accuracy/
[2] https://aerialservicesinc.com/just-how-accurate-is-lidar/
About the Author | Vatsal Srivastava
Aspiring Data Scientist and Self Driving Car Enthusiast
