Objective: Detecting lanes is the most fundamental feature of any ADAS which assists the autonomous vehicles or drivers to stay on lanes while driving. In this project a pipeline was developed to detect lane markings on road from an image and fine tuning the parameters to extrapolate the detected lane markings for improving the lane detections on the road in a video stream recorded by a vehicle which can also be implemented in real-time by mounting the camera on a vehicle.
Implementation:
The lane lines tend to appear in a common region of interest when the camera which is mounted at a fixed position of the vehicle pointing the front side downwards. Due to which the right and the left lane lines appear slanted. As a first step, we load the image or the video stream.
Thus creating a pipeline to detect the line segments in the image, then average/extrapolate them and draw them onto the image for display . We can also feed a video stream in the pipeline to detect the lanes.
Gray scaling
Apply the Grayscale transform on the image. Gray scaling will convert any image with more than one color channel and return an image with only one color channel(gray).
The gray scaled image was then converted into a binary(black/white) image.
Gaussian blur
This is one of the pre-processing techniques which blurs or smoothens the edges of an image by applying gaussian blur on it which acts a low pass filter suppressing images’ high frequency component thus reducing noise by smoothening the image.
Implementing Gaussian Blur in OpenCV take a integer kernel parameter as the intensity of the smoothing. Gaussian blur is used to improve the results of the edge detection followed in the next step.
Canny edge detection:
It is edge detection operator applied on an image to obtain essential structural information(edges) of an image and discard all other data.
It is a widely used algorithm in most computer vision applications which requires edge detection.
Its implementation in OpenCV requires two additional parameters-a low and a high threshold to detect only the edges with intensity lying in the threshold. Edges below threshold are discarded.
Region of interest
Our image frame now consists of only edges of the objects in the image which includes all of those present in front of car along with the lane markings. Thus it is difficult to detect the lane lines automatically in presence of other objects.
Thus we extract and use only a segment of the edge detected image by defining a region of interest by guessing the boundaries of the polygon for ROI, in which the lanes would be generally visible.
We define the ROI with an assumption that the position of camera is fixed and remains the same for all images.
Hough transform- to detect lane lines:
For extracting the lines and highlight the lane lines we apply the Hough Transform technique. Lines are detected by identifying the points lying on them for which there is a need to converting system(x,y) to Hough space (m,b) which uses polar coordinates . This is done by converting our current system denoted by axis (x,y) to a parametric one where axes are (m, b). In polar coordinates, a given line is expressed as (ρ, θ), where line L is at distance ρ and angle θ from the origin, thus making right angle ; that is ρ = x cos θ + y sin θ.
All straight lines passing through a point corresponds to a sinusoidal curve. Thus points lying on the same straight line in cartesian space yields sinosoids which will cross at point (ρ, θ).
Hough transform is widely used in computer vision applications for feature extraction.
Read more on Hough Transform
Gradient Interpolation and Line Extrapolation
The right left distinction was done using the slopes of the found Hough lines. Upon right left categorization based on slope the many left and many right lines were merged into one each by calculating and averaging their intercepts.
Results:
Implementation on a video stream: After modifying a the code a little bit for Hough transform and slope calculation and line extrapolation the following result was achieved.
Challenges: Lanes lines at the edge of road which were closer to the sidewalk were not detected due to the unstable movement of car thus the half of the lane went outside the region of interest for which it remained undetected. Increasing the margins of ROI made false detections with the edge of sidewalks. By obtaining a good contrast on the new ROI obtained after many trials of changing Hough parameters the pipeline was able to mark the lanes accurately.
Future Scope
Overall, there are many possible improvements to be explored but a good balance of necessary update rate of the pipeline and complexity of the pipeline needs to be found under constraint computational power available in the car. The project is implemented offline on videos that already exist but in an actual car would have to be done in real time. If the pipeline takes too long to process individual frames for the computations being time consuming, the car might travel further until the image is processed. to have a high update rate of processed images, maybe dependent on the speed of travel and dynamics of the traffic situation. The pipeline can be made more robust by warping the detected lane boundaries and then using sliding window technique to detect lanes.
