To shift our self-driving cars into the fully autonomous fifth gear, we must design them with the capability of mapping out their environments.
Cars designed to meet the third level of conditional automation are already doing this: their data-collecting sensors continuously modelling both static and dynamic objects in the environment, from street layouts and parked cars to pedestrians, cyclists, and other hazards we encounter on roads every day.
A fully autonomous self-driving car requires four complementary external visual recognition sensors
Each visual recognition sensor has strengths and weaknesses. Only by working in tandem, in what we call sensor fusion, can we can ensure safe and accurate environmental modelling.
Sensor fusion is the first of four pillars needed to support full automation for vehicles. To achieve it requires that these four sensors work together:
- Camera
- Radar
- LiDAR
- GPS
Now we’ll break down what each sensor is and look at what it does.
Working in RGB, the humble camera provides high-resolution visual recognition
In driving, the camera, or passive visual, is the sensor most similar to the human brain. Of the four sensors, it’s the only one capable of reading a traffic sign — the result of machine-learning technology: the neural networks trained on tens of millions of ground-level image sets composed of various street signs across different countries.
Deep reserves of data currently make deep learning tech more actionable for cameras. But there are still problems. Cameras have the same problems as human drivers. They struggle when there’s poor visibility (illumination variability) or extreme weather and aren’t great at depth perception. To counteract this requires developing more sophisticated algorithms, which can arrive at better estimates, or entrusting in sensor fusion. One way of balancing out a camera’s visual recognition limitations is by using radar.
Radar is nothing new; we already use it for parking, checking blind spots, and emergency braking.
Because sound and radio waves are resistant to interference, these sensors boast the greatest acuity when it’s dark or foggy and raining heavily. Radar also has a greater range than LiDAR, making it more suitable for larger vehicles with greater stopping distances.
But it can lack angular accuracy, especially for objects on curves. Radar also has the advantage of being solid-state and durable — the economic alternative to LiDAR.
LiDAR adds 3D modelling and depth accuracy. But it comes with a cost. Is the price worth paying?
A LiDAR sensor uses light to map out its environment, firing up to 900,000 lasers per second at a target and measuring the time it takes for them to bounce back and return to the source. It’s light agnostic, working just as well at night as during the day. Here it is in action:
There are two types of LiDAR — scanning (mechanical) and flash (solid state). Each has pros and cons: scanning is higher resolution but produces more motion distortion. The main advantage of LiDAR is its depth accuracy.
LiDAR divides opinion among companies. Waymo loves it, but Tesla has turned its back on the technology, relying on cameras and radar to do the same work. Tesla’s decision comes more from economic pragmatism than deep conviction. Despite recent efforts to make it more affordable, LiDAR is still expensive.
To cut costs, Tesla would rather rely on their radar and camera sensors — and hope they’re fed the right data. This certainly seems the easy option. LiDAR brings its own challenge in the labelling of 3D Point Cloud data. As to who’s right, only time will tell.
Because the speed of light is constant, LiDAR sensors can accurately measure the vehicle’s distance from any object — using Distance = (Speed of Light x Time of Flight) / 2. What isn’t constant is the location of the car. Hence the need for a GPS system working in tandem to track the point where the laser was fired and where it was received.
Though lacking in accuracy, GPS plays an essential role in localization
High-precision GPS systems form part of the sensor fusion that makes driverless cars a reality. The problem? Because of atmospheric, geometric, and design factors, the accuracy of the satellite signals isn’t necessarily reflected in the signal you receive. GPS accuracy isn’t yet where it needs to be to avoid collisions and still needs visual odometry backup.
Each external sensor has its own pros and cons. Cameras produce high resolution but fall short with depth perception. Radar has the range, even in adverse weather conditions, but lacks the accuracy. And LiDAR performs strongly across the board, apart from proximity detection and in extreme weather, but currently comes with a price tag. The best solution for now is to combine the strengths of each sensor to cancel out each individual shortcoming.
Visual recognition sensors don’t just model the outside environment. They monitor the state of the driver too.
One often overlooked component is the sensor inside the vehicle that detects the state of the driver. We’ll be looking more at driver state detection sensors in a later post. Suffice to say for now that they can monitor the number and position of passengers to estimate how long it might take a driver to regain control of the vehicle, or to help activate the airbag.
It’s no secret that the future of automated driving lies with sensor fusion. “You need a combination of cameras, radar, and lidar in order to create a self-driving system,” says Jada Tapley, Aptiv’s VP of Advanced Engineering.
The continuous data collection by the sensors, the use of this data in trainings and simulations to update maps, SW, and DNN, and the feedback to the software will be critical in improving automated driving.
And the race is certainly on. Human error accounts for 94 percent of crashes in the US.
With smart visual recognition tech working together to create sensor fusion, at least where safety is concerned it won’t be long before self-driving cars overtake us.
Sensor fusion creates the conditions for self-driven cars to understand their environments. This is the first stop on the road to full automation. The next is the drawing up a virtual map and locating the car within it.
To learn more about ingedata.net, drop us a note at datateam@ingedata.net.
Originally published at www.ingedata.net.
