Self-Driving Car: Steering Angle Prediction
The problem of self-driving cars is interesting but complex as a whole. Apart from various ways of inputs, the complexity of the problem lies in variability in external factors such as environment (sunny, cloudy, rainy, etc. ), the density of pedestrians, nature of the road, etc. Due to difficulty in all these, it can be a pinnacle in Artificial Intelligence if we can solve this problem end-to-end. In the end-to-end self-driving car problem, the inputs can be in the form of a video sequence from front camera and rare camera, GPS signal to enhance positioning operations, LIDAR (Light detection and ranging) to have a better idea about lanes and edges of roads, etc. Following diagram that can explain all these functionalities given below.
Once, all these inputs are received by the computing box, various decisions have to be taken by a self-driving car such as steering angle, braking system, acceleration, etc. In this project, we’ll address a simplified minimal problem to handle a software part of self-driving cars which is the prediction of steering angle given a sequence of images or a video. Also, we will see how a deep learning-based algorithm performs the job for us. In this problem, we will have a front camera view as given below and we need to predict a series of steering angles based on a series of images.
Here, we will put it as a regression problem where we give a single frame to the model and the model has to predict the steering angle. One can argue that a series of past images and the current image can perform better. But experimentally it is found that both approaches give a similar performance and in fact, later approach causes over-killing of the problem. Hence, we will use CNN based deep learning model where CNN performs the feature extraction and the final image representation regresses on the golden steering angle. This work is based on the research paper from NVIDIA. We will go one step further where we build a Streamlit based web app to show the effectiveness of the model.
The diagram besides gives an overall idea about the model architecture. It shows the number of CNN layers, the number of fully connected layers used in the model. The model implementation is in Tensorflow. We used the dataset given at the GitHub profile of SullyChen. The dataset contains around 45k images which are sequential images of a few min drive.
Experimentally settings: In each layer, we perform 3D CNN with 5 *5 kernel matrices with the addition of bias. This follows the ReLU activation function. We perform 5 CNN layers and 4 fully connected layers. In fully connected layers, we keep a dropout of 0.5. Ar the end, we keep a single neuron with a linear activation function. The output is a scalar value and will be used later to regress with a golden steering angle. We calculate mean squared error and add L2 regularizer with a weighting factor 0.001. We train the model with Adam optimizer with a learning rate of 0.0001. We train the model for 30 epochs with batch size 100. We split the dataset with a 70% training set and a 30% testing set. The following video shows the performance of the testing dataset.
To add further, we have developed a simple web app using Streamlit. With Streamlit, we can create interactive apps with various functionalities. Here, we simply evaluate our model on the testing dataset and show the performance of the model with a visual representation.
In this short article, we tried to show how a steering angle prediction (subproblem of a self-driving car) model performs on the testing dataset via a streamlit-based application.