Designing a Robust LiDAR Vision Software for Autonomous Driving
Written by Jaeil Park and Thorsteinn Jonsson at Seoul Robotics
In recent years, LiDAR-based perception systems have seen an increase in popularity alongside the development of self-driving cars. At Seoul Robotics, we strive to provide cutting-edge solutions in autonomous perception technology, and we are proud to introduce our latest perception technology for autonomous driving applications.
Our LiDAR Vision Software is a perception software capable of ground segmentation, object detection and classification, tracking, and prediction. LiDAR point cloud processing is notorious for expensive computational costs, but by utilizing edge devices, we are able to deliver performance that works accurately in real-time applications.
Ground Segmentation
The first step in processing the point cloud is ground detection. In almost all cases, the drivable area a vehicle can traverse is a continuous surface and based on this fact, we apply a stochastic model-based fitting to find a model for the ground estimation. Our ground segmentation method can learn the ground shape of various driving cases in real-time and classifies each point as ground or not. The ground height is estimated over all of the observed space with a mathematical model to increase the accuracy of object detection and to make up for occlusion in the point cloud.
Object Detection
Our Deep Neural Network detects objects which have to be carefully tracked while driving. It detects people, cyclists, cars, buses, trucks and more with single shot inference in real-time. In late 2018, our latest Deep Learning model, the Hierarchical Feature Detector, was among the top seven in one of the world’s most renowned object detection benchmarks, the Kitti Object Detection Benchmark. Among the top ten algorithms, ours was the only one that could reliably detect objects in real-time (20+ frames per second) solely based on LiDAR point clouds as others also relies on cameras for detection.
Although our Deep Learning-based object detection method detects objects very accurately, it cannot detect objects which are not in the training database. It is impossible to incorporate all objects of interest in the database, so to overcome this limitation our object detection method uses an unsupervised 3D feature extraction-based method alongside the data-driven method. Object proposals are made with both methods in parallel and the results are fused. This hybrid object proposal assures that the object detection does not miss any objects even though some of them are not classified as the pre-defined classes. Furthermore, this enhances the result from the data-driven method with the exact 3D features from the point cloud input.
Object Tracking
Powered by advanced Kalman Filtering techniques, our solution offers accurate tracking of objects of interest such as cars and pedestrians. If the machine information from the ego-vehicle is available, the vehicle’s odometry can be estimated and used to calculate an accurate representation of the movement of tracked objects around the vehicle. This means that the velocity of each tracked object can be estimated in a global coordinate system, rather than only in the vehicle’s frame of reference.
Another benefit of our tracking system is that it can improve on the object detection result by providing a history for each tracked object. This is especially useful for sparse point clouds (due to distance, occlusion, sensor limitation, etc.), as the object’s history can provide critical information that can not be inferred from a single frame. This results in a much more accurate estimate in terms of position, size, orientation and even classification of the tracked objects and proves to be important for accurate prediction.
Hardware Agnostic
All our algorithms, including our Deep Learning-based inference, are designed to be sensor agnostic. We train and test our algorithms with a variety of point cloud data, both real and simulated, to cover as many different types of scenarios as possible. In addition, we apply intensive data augmentation by randomly transforming the point cloud in the training process. This ensures that our algorithms are not affected by the position and orientation of the sensors so that users can attach LiDAR in any way they want. Our algorithm captures unbiased 3D features from the point cloud for robust object detection with many different forms of point clouds. Our testing shows that we can perfectly cover LiDARs from Velodyne, Ouster, Hesai, Robosense and more.
We are hiring. Check available position here!
