Robust Localization With One Sensor — Part II
By Scott Harvey, CTO and Satya Vakkaleri, VP Product
To LiDAR or not to LiDAR, that is the question
In the previous blog post, we introduced the concept of using a single sensor for localization. Before jumping into the details of the Civil Maps solution, we would like to take a moment to celebrate the news of Volvo’s next generation of cars to be equipped with LiDAR. It is not surprising that a car manufacturer known to put safety first has bucked the industry trend and decided to use LiDAR.
In the last year, Civil Maps doubled down on its hypothesis that LiDAR would become a mainstream sensor in the future. We continued to actively pursue our product roadmap to build LiDAR only localization. Since then, we have seen a major pull from the delivery market with bots equipped with LiDAR. Now, we have a major OEM making a historic move to include LiDAR in their next generation of cars. While it is too early to make a definitive prediction and celebrate along with the LiDAR manufacturers, we definitely see promising signs in LiDAR’s future. Its acceptance in other non-automotive applications such as AR will only push innovation of the LiDAR sensor further, and simultaneously lower the sensor unit cost in the years to come.
Industry Approach for Localization
Now, back to Part II of the blog. If you have not read the previous post, we strongly recommend reading it here before proceeding.
In order to provide localization with just a rough GPS, a global correction system must be robust to GPS error. Let’s take a look at a couple of the common approaches to localization that uses reference data:
- Use of raw point cloud as the reference and applying traditional methods like ICP: In this approach, the matching algorithm uses an initialization position and relies on relative matching of the two point clouds, one each from the reference and the query. The closer the initial estimate, the better the matching algorithm will work. This type of approach breaks down when there is not a high end GNSS system available to provide the initialization position, or when the system is in an urban canyon and the GPS as a result is highly noisy. The system could be initialized near a local optimization minimum and not the correct global minimum. In addition, the relatively large size of the point cloud data (several GB per km) requires a large amount of storage space and high compute, making it cost prohibitive for large scale commercial deployment.
2. Use of High-Definition Map (HD Map) as reference data for localization: In this method too, the matching algorithm uses an initialization position and relies on matching the objects detected in the environment with the features in the HD Map. The design of the system is inherently fragile. Since this method also requires an initialization position, it faces challenges similar to the one described in the method above. Lastly, if the supplier of the HD Map does not meet the high accuracy requirements, or the HD Map does not reflect the latest road environmental changes, then this could lead to a loss in localization accuracy as well.
Civil Maps Solution
Civil Maps’ fingerprinting matches hashes generated from the reference and query point clouds instead of matching between point clouds for the initial global correction. By matching georeferenced hashes, the fingerprinting allows a lookup of the location, instead of an optimization problem that requires an accurate initialization. The added benefit of comparing hashes generated from a region of points is a massive reduction in the size of the reference map used for localization; the encoding converts 3-5 GB of raw sensor data per km by 1,000x. The small data footprint of the fingerprints allows for faster download from the cloud and less local storage and memory requirements.
LiDAR Odometry
LiDAR odometry is a key enabling module for LiDAR-only localization. LiDAR odometry compares neighboring LiDAR scans to estimate the change in position over time. Across our ground truthed datasets, we have been able to achieve lower drift and compute usage than the top implementations from the KITTI benchmarking site. Drift is the error in the odometry estimation and affects the error of the final localization output.
Since the estimation process is in a local frame, a global correction method is needed to remove the drift and transform the trajectory into the global frame and put the sensor on the map. In order to perform a global transform, the LiDAR odometry creates a point cloud buffer that is matched against a reference fingerprint map, generating global corrections to the LiDAR odometry. These global corrections are fused with the LiDAR odometry to produce an output trajectory. An estimated confidence is outputted with the output trajectory and is a function of the distance travelled since the last global correction as well as the estimated accuracy of the last global correction.
Once an initial global correction has been made, the fingerprinting switches to a local lookup where it uses the previous localization result. This process further improves localization confidence as the localization is providing consistent spatial results across neighboring frames.
Our primary output is a global localization, but we also provide just the LiDAR odometry output if there is no reference map available. The localization output is 10hz, matching the standard LiDAR scan frequency.
Docker Deployment
Civil Maps packages its real-time system in a docker for simple integration. The packaged docker contains the dependency libraries as well as the C++ based binary that runs the localization. The docker is run with a mounted directory (for cached data on the host, including the reference map or recorded sensor data), network configuration to the host as well as /dev if USB GPS devices are used. The docker can also take configuration parameters that can change the modes of operation including data collection (logging sensor data to disk), LiDAR localization with prerecorded sensor data playback, or LiDAR localization with live sensor data. Civil Maps’ runtime docker runs on both x86 and ARM64. As a note, it is quite interesting to see NVIDIA thinking along the same lines with containerization at the edge.
Sensor Integration
Civil Maps’ LiDAR localization requires the packet stream from the LiDAR as well as a rough GPS to look up the global localization map. Since the localization system does not require additional sensors like an IMU or Wheel Odometry, the issues with integration and calibration of sensors to each other are bypassed. Civil Maps currently supports many different USB GPS systems as well as WiFi or cellular GPS lookup using Geolocation api. Alternatively, GPS samples can be sent to a websocket server.
FAQ
Our sensor suite has cameras with no LiDAR. Do you have a camera based localization product?
At CES this year, we unveiled our single image localization from publicly available dashcam images. Our initial use cases for camera based localization are focused around change detection for HD map updates.
Does Civil Maps localization technology depend on a particular brand or model of LiDAR?
No. We have certified this technology for a range of LiDAR sensors including Velodyne, Quanergy and Ouster. If your sensor suite uses a different brand of LiDAR, or if you are a LiDAR manufacturer interested in getting your sensor certified, contact us.
A LiDAR agnostic, low storage, low compute solution that delivers accurate localization — this sounds too good to be true. Can you give us a live demo?
Glad you asked! Sign up here and we will schedule a live demo with you in San Francisco. In the meantime, enjoy this recording.
