Why are we planning to use deep learning and Cascade Particle Filter for Indian road lane detection ?
Road lane detection is one of the important part for any autonomy system. After successful validation of object detection model on India road with local objects, we started working on lane detection (Day and night), lane generation and width calculation between the road edge and road lanes.
We started with image and video processing. Since there are many open source examples available on how to detect the road lanes. We also started following the same methods but after applying those methods we were unable to get the kind of result which we were looking.
Till this time we understood that detecting road lanes on Indian road during day time is not going to be an easy task. But parallelly one of our team “Moto Dream” member were working on road lane detection during night time and road lane width approximation. He suggested to use Gamma correction.
On Indian road due to absence of street light, night videos are very dark which makes road lines almost invisible.
So for increasing the road lane visibility we tried few methods. In all methods which we used, we split RGB images to three channel and applied method for each channel then combine them into one image. Among all three Gamma correction result was acceptable at an extent.
1)Histogram Equalization on Hue channel of HSV color space: Looks good but still dark.
2)Histogram Equalization on each channel of RGB image: The brightness of output image was very high.
3)Gamma Correction: Gives best result
As a starting point we chose gamma correction parameter as 0.29, meanwhile we are working on “Adaptive Gamma Correction” and you will see larger improvement in our future results.
We are not using gamma correction every time. We need gamma correction only for night time. So we wrote basic function for binary classification.
Firstly we convert RGB image to HSV image then split three channel (h,s,v). Later taking ‘v’ channel of image (value channel — shows the amount of brightness of the pixel) and count non-zero pixels (with cv2.countNonZero()). Thus we can find zero values ( 0 pixels ) with subtracting non-zero pixels from total pixels.If this result exceeds the threshold value, return True (for night). This method looks good only for our dataset. Later we can decide day or night with SVM.
2. Define our region of interest
We crop the image with Region of Interest mask in order to get rid of objects at the top and at the sides of the roads. We don’t want them to influence the outcome, since we only care about the road.
We knows that the lanes will be located in the lower half of the image, usually in a trapezoid covering the bottom corners and the center. We don’t want our region to be too narrow and have the lines out of our region of interest.
3. Reduce the noise and make data smoother
The video can be noisy and can cause false edges to be detected. We want to avoid it. The solution is bilateral filter algorithm that reduces the noise and preserves edges, crucial for our algorithm to work.
4. Color Thresholding
We now want to filter out (turn black) any pixel that is not the color of the lanes. Lane lines are either white or yellow on roads. Color Thresholding is a perfect solution to discard objects of other colors.
5. Apply Canny Edge Detection algorithm for finding edges
After color Thresholding, we move on to the Canny Edge Detection algorithm that look for places with great color gradient (the bigger the change in the color of adjacent pixels, the bigger the gradient). We have to set 2 thresholds. If the gradient greater than the upper threshold, it marks this point. If it’s smaller than the upper threshold and greater than the lower threshold it will check whether the point is related to some other edge. If yes, mark it. Otherwise discard this point. It is especially useful in detecting lane lines, because of their usually significant contrast with the road.
6. Apply Hough transform to detect lines on our data
Hough transform allows us to easily extract all the lines passing through each of our edge points. It’s basically a brute force algorithm that tries all possible lines intersecting marked points in the image.
It very often detects horizontal lines that are not desired. That’s why we applied some maths to calculate the derivatives of all the lines and discard those with small slope.
7. Apply Clustering algorithm to draw clear line
The output of Hough Transform algorithm can consist of too many lines (usually over 100). If we draw all of them we would get a really messy result. We decided to plot the lines’ parameters and noticed that there are a few clusters noticeable.
We used DBSCAN, it reduces noise by not taking into account lines that don’t occur in bigger clusters.
Going ahead we have planned to use Particle filter (Cascade Particle filter), Kalman filter and recursive Bayes etc. And we are hopping to get some exciting output.
Road Lane detection using Deep Learning
As you can see in both the output result we were unable to get the kind of result we were looking (We are keep working and output will increase with time). So, one of our lane detection team (Gray Matter) decided to use Deep Learning for road lane detection. And the result was surprising. Because we used existing model and trained with some other data. But the model was able to detect the road lane for most of the time and also able to change the lane as video frames changing the lane.
We used Neural networks more specifically Convolution neural
networks. The architecture comprises of few convolutional layers followed by
deconvolutional layers along with Max Pooling , Upsampling and Dropout.
Since we had no plan of using deep learning for lane detection that’s why we have not annotated the any data for road lane detection. So we decided to use the available training data-set and respective annotated data. After data preparation we feed the training data into the model and started the training. As always we were excited to see the output and the output was surprisingly many fold better than what we were expecting.
This small deep learning experiment helped us in understanding on deep learning might be more useful on Indian road for lane detection. Now we have started building our own CNN model and annotating our local data as well. We have set our time-line before Christmas to train the model with the local data. So, you can hear from us very soon with super efficient output.
We have open source our first 2000 local images with (25,000 labels). You can request for data set Click Here
If you have any suggestions for us please let us know in the comment. And if you want to join our team share your profile at career@giscle.com
All these work has been done by the Team: Moto Dream (Olaf Placha, Ahmet Yaylalioglu, Prashant Shukla and Prashant Singh) and Team: Gray Matter (Pratik Agrawal, Hardik Tiwari, Aditya Karhe, Pankaj Shaw, Chaitanya Dandvate)
