This past winter, our team hit a major milestone. Our driverless vehicles surpassed 10 million kilometers (more than 6 million miles) in autonomous mode. The majority of these kilometers were driven in Moscow, one of the most challenging cities in the world with many kinds of bad weather throughout the year.
Difficult weather conditions pose a challenge for self-driving vehicles. They not only reduce visibility but also make road surfaces more slippery. These conditions create different traffic dynamics and thus impact the cars’ decision-making on the road. Since starting our driverless project, we have never taken any shortcuts. We have always been focused on creating a system that can perform in all weather conditions.
In Russia and abroad, this past winter was one for the record books. On some days in Moscow, snowfall amounts broke 50-year records. But despite these difficult conditions, our driverless cars operated throughout the whole winter season. Today, we’re going to talk about the impact winter weather has on driverless technology, and how we ensure that it performs flawlessly even on the snowiest of days.
Filtering snow from the lidar point cloud
Snow immediately brings one issue to mind — impaired visibility. As some of the lidar laser beams reflect off the snow, they are prevented from reaching their target. Fortunately, our driverless vehicles generate over a million beams per second, guaranteeing that a significant number reach their intended objects, even if some are reflected. We enhance the lidar performance in the snow by implementing neural networks to filter snow from the lidar point cloud which enhances the visibility of objects and obstacles around the vehicle. Take a look at it in action:
Exhaust fume condensation
The winter weather also creates another specific challenge — condensation clouds created by car exhausts and heating vents. In cold weather, strong condensation can look like solid obstacles in the lidar point cloud. Once again, we turn to neural networks to help resolve this issue.
Using millions of kilometers worth of winter driving data, we train neural networks to recognize car exhaust fumes. This information is then fed into the planning system, so that it knows to ignore these fumes when formulating driving routes. This helps our driverless cars function correctly, even when surrounded by heavy exhaust and heating vent condensation clouds. The video below shows how the system works by detecting and filtering these fumes from the lidar point cloud.
When comparing the visual camera view to the lidar point cloud, the advantage of the filtered lidar data is clear, with a pedestrian on the edge of the road detected by the system even when standing behind a cloud of fumes. It would be very challenging for a human driver to spot pedestrians in these conditions.
On top of atmospheric challenges, winter also provides road-level ones. Snow comes in many forms — it can be fresh and loose or hard and compact. It can appear harmless to the eye but be hiding treacherous ice beneath it. Snow conditions can change along one stretch of road or throughout one day. To guarantee a safe driving experience, our self-driving system must be able to adapt to these different road conditions.
The key to this is the friction coefficient between the road and the car’s tires. This coefficient is automatically determined by our self-driving system and impacts all of the car’s decision-making. For example, it allows the system to determine how quickly it can accelerate based on road conditions, and thus it can make a proper decision regarding changing lanes — whether to switch right away or wait until a better time. It also has a role to play in braking, constantly determining the safe distance from the car in front and also evaluating the right time to start braking before traffic lights.
Other important decisions, such as maneuver trajectories or maximum safe speed, also depend on weather conditions. Getting all of these details right not only ensures a safe journey in tricky weather conditions but also guarantees a more pleasant and comfortable experience for passengers and other drivers around our cars. This is particularly important when driving in dense urban traffic. Below, we have shared a video showing one of our many autonomous trips along Moscow’s snow-covered roads.
Heavy snowfall can change the landscape beyond recognition. It obscures road markings, and sometimes even the boundaries of the road. On some streets, particularly those used to store snow cleared from the roads, snowdrifts can be the size of buildings. This winter, our localisation technology has proven its robustness and reliability.
Our automated, self-updating maps system enabled us to adapt in real-time to changes in the city. Even before our maps are updated, our driverless cars continue to localize themselves and navigate seamlessly through the streets. Thanks to our sensor fusion technology’s ability to combine data from lidars, inertial measurement units, and odometry, our vehicles are able to localize themselves even in situations where the car slips or skids. The pictures below show a snow storage street both before and after heavy snowfall.
There are a large number of hazards that also don’t just disappear in the winter. In fact, they sometimes become more complicated during winter weather conditions. For example, there is usually a significant increase during the winter months in the use of road maintenance equipment. In addition, snowdrifts narrow the streets, resulting in fewer available parking spaces. This increases the number of illegally double-parked cars, which makes it very difficult to keep normal traffic flowing. This increases traffic jams and incidents of dangerous driving as impatient drivers try to avoid them by moving into the oncoming lane. We can’t forget about pedestrians either. On particularly cold days, they love to take shortcuts to get home or to the office quicker. These shortcuts usually involve crossing in the middle of the road, rather than using designated crosswalks. Take a look at a typical city center journey after a heavy snowfall.
Our aim is to create a universal self-driving technology that can be safely used in different cities and countries around the world. We cannot achieve this without testing this technology in varying and challenging weather conditions. Moscow has a harsh seasonal climate and experiences a range of difficult weather conditions throughout the year. By testing our system here, we are giving it the tools to work in regions across the world, regardless of the climate.