Self-driving Cars are New Rocket Science
We use the phrase “rocket science” to denote a system or activity that is extremely complex. As history shows, it is very hard to design and build a vehicle that accelerates to 7.8 km/s (orbital velocity needed to maintain a stable low Earth orbit) and have precise control on its trajectory. Anatoly Levenchuk argues that it is much harder to control a vehicle that moves safely on real roads with 300 times smaller speed, 25 m/s (90km/h), near other vehicles with similar speeds. It gets even harder if you want a cost of the solution to be close to a price of a car, not of a rocket.
Huge complexity is in algorithms for computer vision, path planning, control. Many of these problems require state-of-the-art deep learning algorithms and large amounts of high-quality training data. But safety is a key. If one sensor fails, there should be other sensors to get the information needed. If a computer fails, it still should do some work to ensure safety. If any hardware or software fault happens, the car should continue safe operation. Hardware and software should conform to ASIL D level of the automotive safety and integrity standard. ASIL D “refers to the highest classification of initial hazard (injury risk) defined within ISO 26262 and to that standard’s most stringent level of safety measures to apply for avoiding an unreasonable residual risk.”
Hardware for computation is an important part. The computation requirements are enormous. The car should handle streams of data from multiple sensors: high-resolution cameras, LIDARs, radars, sonars, GPS. It needs to localize the vehicle within centimeter accuracy, detect cars, pedestrians, bicyclists and other objects on the road, estimate their velocities, plan a safe path for the car and control the car hardware.
Many autonomous car companies still use powerful commodity hardware in their test vehicles. They put one or more PCs with powerful general-purpose GPUs in the trunk. It is a cheap way to start testing. I took a photo of one of such testing cars on NVIDIA GTC 2017 conference.
Autonomous car prototypes use around 2,500 watts of energy, according to Wired. It leads to increased fuel consumption for ICE cars and reduced range for EV vehicles. Apparently, more power efficient computation is important for the future of self-driving.
Intel, Tesla, Qualcomm are building their chips for self-driving cars, but NVIDIA is the leader in this space. NVIDIA DRIVE Xavier is SOC with 8-core CPU and 512-core GPU, a deep learning accelerator, computer vision accelerators and 8K video processors. It’s delivering 30 trillion operations per second (it’s not clear what kind of operations, though) and consuming just 30 watts of power.
NVIDIA DRIVE Pegasus is a system that contains two Xavier SoCs and two Volta GPUs. It delivers 320 trillion operations per second with 400-watt power consumption.
Car computers should not only be powerful, small and efficient. They must conform to the highest automotive-grade standards of reliability. NVIDIA designed Xavier and Pegasus for ASIL-D level certification.
Designing such hardware is very hard and expensive. Xavier was built with 12nm process. It contains 9 billion transistors. They spent $2 billion on development: it took four years of work of 2000 NVIDIA engineers to design the system. It is the most complex SoC chip that the humans ever built.
NVIDIA presented details of their automotive technology stack at CES on 1/7/2018.
Besides Xavier SoC, it includes the DRIVE OS, a real-time operating system, also certified for ASIL-D. It is based on Blackberry QNX and uses application safety framework by TTTech.
The presentation video:
Check out the slides with technical details.
Anatoly Levenchuk notes that the complexity is on many levels:
- SoC chip with 9 billion transistors on a 12nm process. Technology for designing and manufacturing such chips is very hard. Software assists in chip design. Chemistry and lithography for manufacturing are very complex as well.
- System software: an operating system, drivers, compilers.
- Application level software: HD maps, GIS, processing data streams from cameras and LIDARs. These are high-resolution streams, each with special noise reduction technology. Deep learning inference.
- Training deep neural networks. It requires huge amounts of real-world training data. Simulators are used to collect artificial training data.
- The hardware of the car itself, as well as the complexity of car factories with their robots and supply chains.
Cars become the most complex devices ever made by humans. Smaller robots will follow.
