We’ve gone to great lengths to ensure we’ve developed a fundamentally safer autonomous vehicle. And we’ve also been diligent about defining what safety actually means so that it’s a quantifiable benchmark we can meet and then improve upon. Now, we’ll detail how we go about testing our vehicles before they’re deployed on community roads so we have confidence in their safe operation.
The order of testing
Our testing program consists of distinct areas that create a deployment approach informed by massive amounts of high-quality data. We always begin with a simulation within a computer environment, then a re-creation at our closed course testing facility in Southern Nevada, and finally a real-world scenario in one or all of our operating areas, and that applies to environments and to our vehicles.
Virtual testing
The more data we have and the more real-world scenarios we can simulate, the safer our vehicles become. We choose to perform the crucial first step of gathering data using drivers in Prius cars to adhere to our safety approach. Those Prius vehicles are equipped with the same sensor stack as is on our zero-occupant vehicles.
The data gathered from the Prius sensor stack is then used to generate a virtual Nuro in a virtual environment like you see in the GIF above. This virtual Nuro bot runs with previously collected sensor data from public roads to measure performance in a variety of scenarios and conditions. This helps us evaluate what might have happened in real-world situations with the autonomy system in charge.
We then can reproduce those scenes by feeding them to our autonomy system running in a real bot on a testing track; our bot will react to agents that don’t have to physically be there. If any scenarios result in a “failure,” we tune our systems and repeat them until the bot achieves a pass, which is when it’s time to move into artificial scenarios.
Artificial testing
This testing involves real Nuro zero-occupant vehicles operating in a closed course environment built to mimic the real world. This helps us enhance the reliability and performance of our service as well as capture events that are infrequent or challenging. We can then test a wide range of different road user behavior, such as their attentiveness, reaction time, and aggressiveness, and it helps us evaluate and improve how our vehicle interacts in traffic under a variety of conditions.
Our test track primarily allows us to control uncommon scenarios in a safe manner. For instance, we can place trash bags or piles of leaves in the road, then collect data about those objects. The perception team can then use that data to train the autonomy system to detect uncommon road hazards it could encounter in the real world. We even use dummies and highly trained people in our controlled environments as actors on the course, which is safer than deploying immediately into the real world with real pedestrians.
All the data run from the closed course will be fed back into large-scale simulation, then scenarios are re-run until the bot “passes” each one safely. We’re constantly increasing operating scope and complexity so that we can confidently advance from theoretical situations to reality.
Real-world testing
In order to get to public roads testing, we validate and test our system in a closed course. Then, if we’re confident the autonomy is ready to proceed, we can again utilize our Prius to conduct on-road test driving with safety drivers that mimic our zero-occupant system in the real world. This allows us to conduct extensive on-road testing with the added safety benefit of a driver behind the wheel.
After we’re confident in the performance of both our simulated bot and our Prius acting in autonomous mode on real streets, it’s time for the most exciting part: deploying Nuro onto real streets. Here, too, we’ve prioritized safety: though we’ve tested and validated hundreds of thousands of scenarios, our bot may encounter a situation it cannot navigate. In that case, teleoperations are able to remotely see what the bot is doing and have the ability to take over control from autonomy. If, for instance, a bot were to encounter a policeman manually directing traffic, our teleoperations team would fully take control of the bot to watch for the policeman’s visual cues, then return the bot back to autonomy.
Ongoing deployment
We take a rigorous, conservative approach to testing and deployment. It’s a long process filled with much repetition and validation, but once we reach the point of deploying on public roads, we’re confident in our vehicle’s abilities to safely navigate the world. And that’s based on a data-backed approach that’s been years in the making.
So, when you see one of our bots in your community, you’ll know it’s navigated those roads thousands of times in simulation, practiced and validated scenarios in our closed-road testing track, and driven with a safety driver as a copilot before the bot takes over. All to bring you the things you need from the stores you love.
