Improving Lidar/RADAR and other photon emissive sensors through data transmission, utilizing Continuous Wave Frequency Modulation (CWFM).
1 WHY IS LIDAR IMPORTANT?
Lidar has emerged as one of the critical components in the quest to build a fully autonomous vehicle. Its provides the spatial resolution required to accurately discern and identify a wide variety of objects and operates at ranges far enough in front of the vehicle that computational models of the environment can be built and meaningful action taken. This combination of features is not available from any competing technology such as RADAR, Ultrasonic or camera vision systems. Lidar does however, have its own set of issues which are currently preventing its adoption by auto manufacturers on mass. These are its high per unit cost, mechanical mode of operation and the correlation between accuracy, range and emission power which has to be limited in order to comply with eye safety legislation. LIRDATA is a new technology which could help to solve these fundamental drawbacks to Lidar and assist with making Lidar the primary sensor option for the Autonomous Vehicle industry.
2 ISSUES WITH LIDAR AND A POSSIBLE IMPROVEMENT
Recent developments in computational power and sensor technology has made autonomous vehicles (AVs) look like they may be viable in the very near future. However, significant hurdles still remain. These challenges can be broadly broken into three categories, first, being able to collect data on the environment surrounding a vehicle at sufficient resolution, range and frequency such that it can be accurately digitized (Data collection). Second, the collation and analysis of that data, so that obstacles and other environmental objects can be discerned and tracked (Perception), and third, to build algorithms that analyze these perceived objects and take meaningful actions based on their presence (Command & Control).
Data collection of the environment surrounding the vehicle primarily relies on four main sensor categories today. RADAR, Lidar, Ultrasonic and computer vision with most AV companies opting to use some combination of these sensors in what is termed sensor fusion. This allows the strengths of one sensor type to offset the weaknesses of another. For example, RADAR can acquire data over a very long range, is relatively cheap, and can see through obstacles such as fog and rain and has no moving parts. But it does not have the resolution required to accurately discern between objects of similar size surrounding the vehicle. Lidar is uniquely placed in that it seems to meet most of the technical engineering requirements required to achieve autonomous driving. That is, it is highly accurate with millimeter precision, it can collect data at ranges far enough in front of the vehicle such that there is sufficient time for perception calculations to be run and meaningful control actions taken. Lidar does suffer from a few drawbacks however, in comparison to other sensor types. It is two orders of magnitude more expensive, it suffers from accuracy limitations at long ranges and typically relies on mechanical actuation to control the laser beam which collects the data, meaning they are prone to wear and tear and have an effective service life of only a few years. Assuming these problems can be solved and Lidar adoption becomes ubiquitous, an additional problem will manifest itself as emissions from one vehicle will be picked up by neighboring vehicles (Crosstalk), therefore increasing sensor noise, with the potential for false positives which may confuse driving algorithms, potentially leading to fatal consequences.
LIRDATA which is an acronym for Lidar-Radar-DATA, seeks to mitigate many of these drawbacks. It utilizes a form of Constant Wave Frequency Modulation (CWFM) to encode data onto Lidar & RADAR emissions allowing independent vehicles to share data between themselves. This offers numerous benefits and helps to solve many of the aforementioned issues. For example, a vehicle can send data to vehicles behind them, therefore extending the effective range of the sensors without the need for increasing power levels. AV manufacturers such as Tesla, which have chosen not to use Lidar due to its cost prohibitive nature, could instead install cheap photodiode arrays for detecting the Lidar emissions of other vehicles and fixed Lidar installations (for example those already installed by governments and private companies at intersections) and integrate that data into their sensor fusion whenever it is available at very low cost. By encoding a unique identifier into the emissions, sensors can also discern which backscattering photons originated from their sensor and which ones emitted from neighboring sensors, eliminating the risk of crosstalk completely. LIRDATA can also be used for velocity detection which is not supported by conventional Lidar sensors using single burst Lidar with short packets. LIRDATA’s CWFM allows for the measurement of the Doppler Shift of the signal, such that the velocity of the target can therefore be determined.
3 BASIC ARCHITECTURE
Vehicle One (V1) contains a Lidar which adheres to LIRDATA’s CWFM standards. Vehicle Two (V2) is some distance behind V1 and contains either a fully functioning Lidar identical to V1, or a much cheaper photodiode array which is capable of reading and decoding LIRDATA emissions from V1.
4 USAGE SCENARIOS
4.1 Sharing Location Data Between Vehicles
V1 and V2 are both are using assisted GPS (and other onboard sensors) for very high accuracy localization and time synchronization. V1 can obtain the distance (d) to an obstacle at t1 using regular Time of Flight (ToF) calculations:
Where delta t, is the measured time between the transmission of the Lidar pulse and the detection of the return signal.
V1 can now convert the detected data point, into an absolute latitude and longitude by taking its own location and decomposing the distance to the obstacle (measured using 1) into its Latitude and Longitude components using the angle of transmission α. This data can now be encoded into the next Lidar pulse emission (at t2) by modulating the frequency of the Lidar’s carrier wave.
V2, which is following some distance behind V1 will occasionally receive backscatter from V1’s Lidar emissions, which contain this encoded point cloud data. V2 can decode this data and use it as sensory inputs into its own autonomous algorithms.
4.2 Sharing Velocity Data Between Vehicles
One benefit of Continuous Wave (CW) technology is that the relative velocity of the target to V1 can also be determined using the Doppler Shift of the Lidar signal using interferometer measurement. The absolute velocity of the obstacle (relative to the Earth) can then be determined by adding V1’s velocity and the obstacle’s velocity relative to V1. This data can then also be encoded onto the carrier wave and also transmitted during the next pulse cycle.
4.3 Sharing Perception Data Between Vehicles
In order to increase the throughput of data, one must decrease the amount of “noise” and decrease the computational burden for vehicles receiving LIRDATA. To achieve this, higher level, perception data could be shared instead of the raw point clouds. This data would be sent only once V1 had fully identified the obstacle determining its size, type, velocity and other characteristics. This higher level perception data would then be encoded onto the LIRDATA stream as previously described. This method, while having clear advantages, would require the formation of a standard for the encoding and encapsulation of perception data/objects that all AV manufacturers adhere to or know how to decode. Such a standard would have many other benefits, such as allowing for plug-and-play between various sensor types used in AV platforms. It is noted that to date, no such standard exists.
4.4 Mixed mode operation
It is envisaged that RADAR sensors should always implement LIRDATA as it is not subject to the same energy constraints as Lidar, as radio waves are not focused by the eye. For Lidar however, it could be envisaged that a combination of both single burst and CW Lidar could be used in sequential operation. That is, if velocity measurement is not required, or is handled by other sensors (such as RADAR) Then the Lidar can be used in single burst mode to determine distance to obstacles at higher power and increased range. Only when velocity measurement or data transmission is required, is the Lidar’s operation switched to CW mode.
4.5 Supporting Other Data Types
The type and amount of data can be easily modified to suit the available bandwidth. For example, anything from light weight, raw sensor data, to entire point cloud arrays or higher level perception based objects can be shared between vehicles depending on the available bandwidth and the specific requirements and desires of Autonomous Vehicle manufacturers.
5.1 Increasing Perception Range & Field of View
It should be clear to the learned reader, the vast array of benefits this data sharing bestows upon V2 which now knows of the presence of obstacles well before it will encounter them with its own array of sensors. Importantly in the case of Lidar, this range extension was achieved without the need to increase laser power levels which are limited by current eye safety laws surrounding laser light emissions.
The ability for the backscattered LIRDATA to reflect off many surfaces and still be detected, means that it is also effective in non line-of-sight applications such as when V1 and V2 are separated by another obstacle or around a corner. Furthermore, having multiple vehicles sensing and detecting obstacles decreases the likelihood of single point failures where one vehicle’s sensors miss an obstacle due to a rare corner case or condition which may have affected that vehicle’s sensors in isolation, at that single point in space and time.
5.2 Reducing Costs
With the implementation of LIRDATA, vehicles which do not have their own Lidar sensors can still benefit from the high resolution Lidar data obtained by their neighboring vehicles or from fixed installations of Lidar sensors which are currently being placed in busy areas such as intersections that are often frequented by AVs. Manufacturers such as Tesla, who have decided to avoid Lidar due to its high upfront cost, may be convinced to install a cheap photodiode array on their vehicles to receive the Lidar data emitted from external sensors so that this high accuracy data can be utilized as another source of high value data, whenever it is available. This allows the benefits of Lidar to be utilized at a fraction of its current cost.
5.3 Reducing Computational Load
Converting point cloud data into accurate object recognition requires significant processing time and computational power. By sharing the perceived data using LIRDATA this burden is lifted from the recipient vehicle, which now has more available computational resources to dedicate to its own sensor data. In this way vehicles in a local area are working together to build the most complete picture of the environment.
5.4 Monetization of Data
Assuming that standard protocols are developed and in use by multiple manufacturers, it is possible to imagine a scenario where a recipient vehicle is charged a fee for receiving and using perception data calculated from a donor vehicle. This revenue could be used to offset the relatively higher cost of the donor vehicle installing the Lidar sensor in the first place and the computational load required to determine the presence of the obstacles.
The sharing of data between vehicles has numerous benefits including increased range, increased field of view, decreased cost and computational requirements on individual vehicles. Future work should be undertaken into the formation of a standard for LIRDATA transmissions which could encapsulate raw point cloud data and/or higher level perceived objects. Similarly, work should be undertaken into the formation of a standard for classifying perceived objects as this will greatly increase the utility of the LIRDATA system.
Nathan Parrott is a Mechatronic Engineer with an interest in Technology, Autonomous Vehicles and Space Travel. You can check out some of his projects at www.nathanparrott.com