Google I/O Recap: Turning self-driving cars from science fiction into reality with the help of AI

By: Dmitri Dolgov, CTO and VP of Engineering

This morning, we took the stage at Google I/O, Google’s annual developer conference, to share details on how Waymo is using artificial intelligence (AI) to make fully self-driving cars a reality. If you’re familiar with our work, you know that AI and machine learning (ML) have played a critical role in moving us closer to our goal of bringing self-driving technology to everyone, everywhere.

Here’s a quick recap of what we shared during this morning’s keynote.

Jump-start in AI: Many people know about Google’s early AI advances in image search and speech recognition. But did you know that Google’s AI researchers also helped give Waymo a jump-start on the road to truly self-driving cars? One example: as deep learning began to take off, our self-driving engineers worked side-by-side with the Google Brain team to apply deep nets to our pedestrian detection system. Even in those early days, the results were remarkable — within a matter of months, we were able to reduce the error rate for pedestrian detection by 100x, making our system safer and more capable on the road.

Fast forward to fully self-driving: Fast forward to 2018, Waymo’s advances in AI have helped turn self-driving cars from science fiction into reality. Today, Waymo is the only company in the world with a fleet of truly autonomous cars on public roads. Members of the public in Phoenix, Arizona have already started to experience these fully self-driving rides.

AI everywhere: AI plays a crucial role in nearly every part of our self-driving system. While perception is the most mature area for deep learning, we also use deep nets for everything from prediction to planning to mapping and simulation. With machine learning, we can navigate nuanced and difficult situations; maneuvering construction zones, yielding to emergency vehicles, and giving room to cars that are parallel parking. We can do this because we’ve trained our ML models using lots of different examples. To date, we’ve driven 6 million miles on public roads and observed hundreds of millions of interactions between vehicles, pedestrians and cyclists.

We use ML in nearly every part of our self-driving system, including perception, prediction, planning, and mapping. Here, we’re using ML to accurately detect pedestrians, even when they’re dressed in dinosaur costumes!

Beyond the algorithm: It takes more than good algorithms to be able to put self-driving vehicles on the road and expand to more cities. Infrastructure plays a key role in training and testing our ML models. At Waymo, we use the TensorFlow ecosystem and Google’s data centers — including TPUs — to train our neural networks. With TPUs, we can train our nets up to 15x more efficiently. We also rigorously test our ML models in simulation, where we drive the equivalent of 25,000 cars all day, every day. With this robust training and testing cycle, we can rapidly improve our ML models, and quickly deploy the latest nets on our self-driving cars.

With simulation, we can rapidly test and validate our ML models, allowing us to quickly iterate and accelerate our software development process.

Weatherproofing our driver: We aim to bring self-driving technology to everyone, everywhere… and in all weather. Driving in heavy rain or snow can be a tough task for self-driving cars and people alike, in part because visibility is limited. Raindrops and snowflakes can create a lot of noise in sensor data for a self-driving car. Machine learning helps us filter out that noise and correctly identify pedestrians, vehicles and more.

In this example, we’re filtering out noise in our sensors using ML, which helps our cars clearly see the road ahead.

With our years of experience, collaboration with Google AI, and powerful infrastructure, we’re getting closer than ever to a future where transportation is safer, easier and more accessible for everyone.