End to End Learning for Self-Driving Cars using python
This project is inspired from Nvidia Research paper of Self Driving. See the link.
Before we go any further, Have you ever thought how do human actually learn how to drive?
The answer to that is, for years and years we watched and observed, we collected data among us and learn the basic of driving. We had model in our head which we trained from the data which we had observe, experience etc.
Abstract
There has been a boom in interest in self-driving automobiles during the last decade. Deep neural networks have been trained to do activities that would normally need human interaction as a result of discoveries in the field of deep learning. CNNs are important in the field of Computer Vision because they employ models to find patterns and characteristics in pictures. Object detection, picture categorization, and image captioning are some examples. I used photos recorded by a simulated automobile to train a CNN in order to drive the car independently in this experiment. The CNN extracts unique aspects from the photographs and uses them to make steering predictions, allowing the automobile to drive without the assistance of a person. The Unity-based simulator offered by Udacity was used for testing and producing the dataset.
Data Gathering
I will be using the open-source Self-driving car simulator provided by Udacity that is used in their Self-driving car Nano degree program. It replicates the original Nvidia simulation. There are two types of tracks and two types mode inside the simulation Training mode and Autonomous mode. Download Udacity Simulation here.
Data Preprocessing
The training data is preprocessed before training the model. I have used data augmentation technique for preprocessing the images. Data Augmentation is a technique that may be used to artificially increase the size of a training set by modifying existing data. It’s a smart strategy to employ for avoiding overfitting, or if original dataset is too small to train on or want to improve the performance of your model. The photos are then transformed and adjusted to fit the model's input shape. Because RGB isn't the ideal mapping for visual perception, this is done. Artificial shifts and rotations are added to the data to educate the network how to recover from a bad location or orientation. We take right, left, or center photos at random, flip them left/right, and alter the driving angle while enhancing. We also vary the steering angle to randomly move the picture horizontally or vertically.
Training the Neural Network
The deep learning model we prepared is based on the research done by NVIDIA for their autonomous vehicle. The model comprises the following important layers. There are 5 convolutional layers with varying number of filters and sizes with activation of “elu” with batch normalization after each hidden layer to prevent from overfitting. In the end, 3 dense layers were added followed by the output layer. Adam optimizer was used for parameter optimization with a 0.001 learning rate and loss function of mean squared error was used. Steps of epochs size of 300 was chosen and the number of epochs of 10 was experimented with.
Solution
Let’s get started with the code. The complete project on Github can be found here. I have used Google Colaboratory but feel free to use which you are comfortable with.
I started with loading all the required libraries and dependencies.
The dataset has 6 columns i.e. center, left, right, steering, throttle, reverse, speed. I have used pandas library reading the datasets.
The next is checking and removing the biased in the data.
The next step was to split the training and validation data. I have used the 80–20 rule which means using 80% of the data for training rest 20% for testing the model on unseen images.
x_train, x_val, y_train, y_val = train_test_split(imagesPath, steering, test_size=0.2, random_state=5)Furthermore, the data is then augmented using augmentation technique. Image are flipped, zoomed, multiply brightness etc.
Moreover, the image is then preprocessed by Converting RGB Image into YUV, adding Gaussian Blur and resizing image.
The next and final step was to build a model itself. There are 5 convolution layer and batch normalization after each convolution layer to prevent overfitting.
At last, the model trained for 10 epochs with 300 epochs size but feel free to adjust the hyper-parameter as you want.
Results
Overall, the model did a good job. The car never goes off track but it appears to embrace the left side of the lane, which is the direction the course's curve is curving. The model shows an excellent performance on critical situation like turning road.
Conclusions
Using deep neural networks, I was able to successfully estimate steering angles in this research. We also demonstrated that CNNs can learn the whole job of lane and road following without the need for human breakdown into road or lane marking detection, semantic abstraction, path planning, and control. A little quantity of data from less than a few hours of driving was sufficient to educate the car how to drive on highways. The system was able to drive safely on the roads where it had been trained, which is an intriguing constraint.
References/Further Readings
Before you go
The corresponding project on Github can be found here.
Reach out to me
Please don’t be shy. Let’s communicate
Thanks for reading and happy learning, and happy coding! See you next time