Image for post
Image for post

Virtual Validation: A Scalable Solution to Test & Navigate the Autonomous Road Ahead

Lyft Level 5
Oct 27 · 5 min read

By: Robert Morgan, Engineering Director and Mason Lee, Technical Program Manager

To build an autonomous vehicle (AV), we need to be able to safely and efficiently modify and test its software and hardware stacks. While on-road testing may seem like an effective way to do this, it’s simply unrealistic. It’s estimated that it would take more than 10 billion miles to collect enough data to fully validate a self-driving vehicle — that’s 400,000 trips around the Earth.

Let’s put this into perspective by considering the level of engineering required for a self-driving vehicle to complete a frequent and intrinsic action for human drivers: lane changing.

Human drivers instinctively consider the variables that impact their decision to change lanes: their own speed, the speed and distance between other vehicles on the road, congestion, whether a lane change even makes sense given an upcoming turn in the route, and others. For an autonomous vehicle to make the same decision, models for each of these variables must be developed and tested over and over (and over) again, resulting in an astronomical amount of time and resources required if solely tested on-road.

Whether making lane changes or executing more complex scenarios, it’s essential to have a scalable, virtual option to effectively validate AVs as we accelerate toward a self-driving future.

Image for post
Image for post

The digital co-pilot to AV road testing

At Lyft, we supplement on-road testing with virtual validation to continuously test our self-driving vehicles’ software and hardware systems. This method involves running a high volume of virtual missions in a representative simulation while measuring performance along the way. We then evaluate the results to make necessary modifications to our software and hardware stacks, and feed the learnings back into the virtual validation system to improve autonomy performance, simulation, evaluation, and test coverage over time.

In addition to eliminating the need to complete miles of on-road missions to gather data for endless scenarios, virtual validation offers a number of other benefits:

Level 5’s approach to virtual validation

Virtual validation is not just a “final exam” we use to evaluate the performance of our software and hardware stacks for real-world deployment. It’s an approach we leverage and fuse with on-road vehicle testing throughout the AV development process.

Image for post
Image for post

Virtual validation is a rigorous, continuous process at Level 5. Every week, we collect our individual teams’ latest releases into a unified software update that we refer to as the “candidate.” Before we deploy the candidate on the AV fleet, it must run through a suite of virtual tests:

Once the stages of virtual testing are complete, the surfaced issues are sent to our Triage team for evaluation. They review the virtual disengagements, failing tests and diagnostics, and monitor the various automated metrics (like road law violations) for each candidate. Once the issues from virtual testing are addressed, we deploy the candidate onto our AVs for physical world testing. This allows us to not only validate the results from virtual testing, but also to feed these new real-world scenarios back into the virtual testing pipeline for future candidates.

Leveraging real-world data to focus on the right scenarios

Virtual validation enables us to simulate specific scenarios at scale and chip away at the target of 10 billion miles driven. But, how do we identify and address the challenging scenarios that require additional testing and virtual validation? The answer to this question lies with real-world datasets.

At Lyft, we have the opportunity to tap into data from our nationwide rideshare network to study real-world driving scenarios that can inform the development of our own self-driving technology. This data can help us focus on what and how we test, like interesting event discoveries such as an object falling out of a truck bed; or naturalistic maneuvers like whether the AV changes lanes in a way that’s comfortable for passengers.

Virtual validation is essential to the development of self-driving technology. It’s safer, faster, and produces more complete answers about the performance and safety of AVs in various scenarios. By combining virtual validation with on-road testing and real-world data, Level 5 can accelerate autonomous driving technology for years to come.

Follow this blog for updates and follow @LyftLevel5 on Twitter to continue to learn about Lyft’s road to autonomy. Visit our careers page to join Lyft in the self-driving movement.

Lyft Level 5

Revolutionizing cars, reshaping the future.

Lyft Level 5

Written by

Level 5 is Lyft’s self-driving division in Palo Alto, Munich and London. We’re hiring! lyft.com/level5

Lyft Level 5

Revolutionizing cars, reshaping the future. Level 5 is Lyft’s self-driving division in Palo Alto, Munich and London. We’re hiring! lyft.com/level5

Lyft Level 5

Written by

Level 5 is Lyft’s self-driving division in Palo Alto, Munich and London. We’re hiring! lyft.com/level5

Lyft Level 5

Revolutionizing cars, reshaping the future. Level 5 is Lyft’s self-driving division in Palo Alto, Munich and London. We’re hiring! lyft.com/level5

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store