Advanced Lane Lines — Challenge Videos Try

For the fourth project of Self-driving Car Nanodegree, our goal is to identify lines using the following pipeline in Computer Vision way.

  1. Calibrate the camera

2. Undistort the image and threshold gradient image to get lane lines clearly visible

3. Choose region of interest and do perspective transformation

4. Identify the line and fit it into the second degree polynomial

5. Draw the area between lines and transform it back to the road

You can find many posts describing this pipeline. I would like to highlight the steps which lead me to success.

  1. Use sanity check to identified lines

When lines are fitted into the second-degree polynomial, we need some sanity check to understand our lines look reasonable. I decided to check that lines are parallel. Actually, I check that lines do not intersect in picture region. Let’s say our left and right lines have equations:

To find points of intersections we have to solve the equation:

If y is not inside warped image borders, our lines are quite parallel and pass the sanity check.

2. Outliers removal

To improve the result I use outliers removal as well. I just remove dots which deviate too much from the line equation and then fit line once again.

3. Use line equation to identify the next line

When the first line was identified successfully, I use line equation with margin as an area to search for the next line. No need to do Sliding Window Search again.

Area to find the next line

4. Line averaging

To avoid line jittering I use weighted averaging over last 5 lines. It is a good question how to average the line. I use following technique.

When our line is identified we generate plotting points for it and use them to display the line on the road. I just combine those dots from the last 5 lines and tried to fit polynomial into this bunch of dots. Since I use weighted averaging I took randomly 18% of dots from the 5th line, 36% from the 4th line, …. and 90% from the previous line.

On this stage I passed the track 1. But the same code outputs awful result for track 2 and 3. I analyzed my pipeline on images from the track 2 and 3 and saw next picture:

The first image from track 3

My first observation that the line is too curvy and my convolution slider is not able to detect these lines correctly.

As result, I decided to rectify it more and end up with a solution when the height of my warped image is 3 times more.

Line angle

Also, I realized that the line could end in the middle of the warped image. At the same time, I have to distinguish it from the situation when the line appears near the image border (line angle is near 90 degrees). To handle this situation I check line angle alpha is “small” enough.

It is easy to see that small noise on threshold images leads to large noise on warped images. My next step was to improve threshold technique. It is obvious that on the third video threshold quality suffer from different lighting conditions. As result, I decided to apply histogram equalization with CLAHE to normalize the image.
I analyzed the road in different color spaces and decided to detect yellow lines from HSV image space and white lines from HSL image space.

On the image below yellow lines are visible on both S and V channels. The intersection of this two channels gives good candidates for yellow lines.

After a close look at L and S layers of HLS, I noticed that white lines are visible on L or S layer. As result, I combined areas from both channels.

Also, I noticed that magnitude and directional gradient thresholds allowed to reduce noise on the image. After all these modifications I got the result:

Conclusion:

Computer vision technique to identify lane lines are really challenging because of different lighting conditions. I tuned my pipeline on two images from the second video and was able to get a nice result. But tuning my pipeline on two images from the beginning of the third track allows me to pass nicely only small parts of the track. You have to play with a lot of examples before you can get more or less reliable solution. Moreover, when I analyzed my pipeline on the images with dark shadows and bright areas, I realized that my threshold is not able to identify lines in such areas.
A threshold is not the only issue I have to handle. I did not pay any attention to the situation when only one line or no lines are visible on a warped image as we can see at the end of the third video.
There is a large room for improvement. In the meanwhile, I switched to P5.