My Google Summer of Code 2021 Journey
About Me
Hey there! My name is Garv Tambi. I am a pre-final year undergraduate pursuing B.Tech in Electronics and Communication Engineering(ECE) from the LNM Institute of Information Technology, Jaipur, India.
I have a programming background as my university courses introduced me to the basics of programming. I developed a keen interest in Application Development. I began programming in my freshman year and have continued it. My main focus at start was in Data Science as well as in Competitive Programming and I used it to develop my coding skills and DS/algorithm implementation. I also have a good knowledge of Computer Vision as well as Deep Learning and have successfully completed various projects.
About the Project
The detection of risk situations during the navigation of mobile robots is an essential task for future applications. The goal is to create a software agent in robocomp with the aim of improving vehicle driving, using deep learning and computer vision techniques.
The main idea is to use one of several RGB cameras placed in a vehicle for lane detection, pedestrian detection, vehicle detection, sign detection and more elements that affect driving. To perform this task, it is possible to work either with real datasets of cameras placed in vehicles or to use the carla vehicle driving simulator.
Pre-GSoC Period
I had been contributing to Robocomp since a few months and have bonded with the community quite well. Also, I was always present in a discussion that was going on Gitter. During this period, my major focus was to make sample prerequisites to build and implement functionalities that I will work on in the main project. Enhance my understanding to get more familiar with the source code of Robocomp, and the community. Discuss with mentors and other community members put together precisely components of my proposal, and include the necessary changes.
Task1: Finding Lane Lines on a Road
When we drive, we use our eyes to decide where to go. The Lane lines on the road that shows us where the lanes are act as our constant reference for where to steer the vehicle. Naturally, one of the first things I would like to do in this project is to automatically detect lane lines using an algorithm.
In this project I detect lane lines in images using Python and OpenCV and later on video. OpenCV means “Open-Source Computer Vision”, which is a package that has many useful tools for analyzing images.
Some steps include :-
- Convert BGR image to Gray.
- Convert into Blurred image using Gaussian Filter(to avoid the noise)
- Apply Canny Edge Detection into the image.
- we define Region of interest(i.e. bottom half of the image)
- Apply Hough Transform and smoothing the final detected lane lines.
For more detailed explanation please refer my post02.md
Task2: Advanced Lane Lines
Now I move towards the next very interesting steps of this project i.e Advanced Lane Line Detection using computer vision. The video below talks about how I process the input stream and the steps involved to build an image processing pipeline that gives filling area between left lines and right lines as well as Curvature as outputs.
Advanced Lane Finding Finding
The goals / steps of this project are the following:
- Compute the camera calibration matrix and distortion coefficients of given images.
- Apply a distortion correction to raw images.
- Use color transforms, gradients, etc., to create a thresholded binary image.
- Apply a perspective transform to rectify binary image (“birds-eye view”).
- Detect lane pixels and fit to find the lane boundary.
- Determine the curvature of the lane and vehicle position with respect to center.
- Warp the detected lane boundaries back onto the original image.
- Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.
For more detailed explanation please refer my post03.md
Task3: Pedestrian Detection
Pedestrian detection in image or video data is a very important and challenging task in security surveillance. The difficulty of this task is to locate and detect pedestrians of different scales in complex scenes accurately.
To solve these problems, a deep neural network (RT-YOLOv3) is proposed to realize real-time pedestrian detection at different scales in security monitoring. RT-YOLOv3 improves the traditional YOLOv3 algorithm. Firstly, the deep residual network is added to extract vehicle features.
Then six convolutional neural networks with different scales are designed and fused with the corresponding scale feature maps in the residual network to form the final feature pyramid to perform pedestrian detection tasks. This method can better characterize pedestrians.
In order to further improve the accuracy and generalization ability of the model, a hybrid pedestrian data set training method is used to extract pedestrian data.
For more detailed explanation please refer my post04.md
Task4: Finding Lane Lines on the Carla Simulator
I run this algorithm on the Carla Simulator that I already ran on real dataset.
When we drive, we use our eyes to decide where to go. The Lane lines on the road that shows us where the lanes are act as our constant reference for where to steer the vehicle. Naturally, one of the first things I would like to do in this project is to automatically detect lane lines using an algorithm.
I run this algorithm on the Carla Simulator that I already run on real dataset.
About the Simulator
This simulation is carried out in Carla Simulator. The CARLA python client runs on Python 3.5.x or Python 3.6.x (x is any number). Python 3.7 is not compatible with CARLA.
For more detailed explanation please refer my post05.md
Task5: Advanced Lane Lines on Carla Simulator
The video below talks about how I process the input stream and the steps involved to build an image processing pipeline that gives filling area between left lines and right lines.
For more detailed explanation please refer my post06.md
Task6: Pedestrian and Vehicle Detection on the Carla Simulator
Pedestrian detection in image or video data is a very important and challenging task in security surveillance. The difficulty of this task is to locate and detect pedestrians of different scales in complex scenes accurately.
To solve these problems, a deep neural network (RT-YOLOv3) is proposed to realize real-time pedestrian detection at different scales in security monitoring. RT-YOLOv3 improves the traditional YOLOv3 algorithm. Firstly, the deep residual network is added to extract vehicle features.
Then six convolutional neural networks with different scales are designed and fused with the corresponding scale feature maps in the residual network to form the final feature pyramid to perform pedestrian detection tasks. This method can better characterize pedestrians.
In order to further improve the accuracy and generalization ability of the model, a hybrid pedestrian data set training method is used to extract pedestrian data.
For more detailed explanation please refer my post07.md
Task7: System Integration on Carla Simulator
The video below is the integration of Lane line detection, Pedestrian detection, Vehicle detection and classifying traffic signs.
For more detailed explanation please refer my post08.md
It is a Wrap!!
“If you do what you love, you never have to work in your life.”
I feel so good at the same time a little disappointed that my GSoC journey is in the wrap-up phase. It was my privilege to work for RoboComp. I started out not knowing much, but in these three months, the amount of learning that happened was exponential. This is probably the most productive summer I have ever had. You can find my contributions to RoboComp here. I would like to thank my mentors who have been supportive and approachable throughout this journey making it an incredible one. I hope I have delivered what I had proposed.
I also managed to opensource my own repositories on GitHub which might be helpful for people who would want to do something similar. My GSoC 20 journey may have come to an end, but I will still be an avid programmer and an active contributor to the community. Hope we cross paths sometime!
Thank You for reading!
