Autonomous Vehicle Industry — To build Trust and Succeed — Do the Opposite of what you are doing
I would like for autonomous vehicle technology to exist as soon as possible. Unfortunately, there are several areas, in whole and part, that will prohibit most of this technology from ever coming to market. And from public trust to ever be gained. In order to remedy this there are several areas the industry needs to do exactly the opposite of what they are doing now in order to reverse the course it is on and actually be successful.
Let me give you the starkest example of opposite. Right now, the entire industry is doing the exact opposite of what it says it want to do. That being to create autonomous vehicles and save lives. Using public shadow driving you will never get close to reaching that goal AND you will take more lives needlessly as you try.
Just last week at CES the new PAVE Campaign was revealed. 25 or so industry entities have banded together to spread the good word and build public confidence. Right off the bat all they did was repeat the current untenable state of affairs or the echo chamber. If autonomous vehicles are literally to ever exist and we want to actually save lives rather than take them almost everything this industry is doing needs to be flipped 180 degrees. The path below is what PAVE should have opened up with and should be championing.
If all of these steps are not taken this technology with never get anywhere close to complete and we will not gain public trust. Think about this — why do we trust the FAA and air travel??????
Training the AI/ML
Current Approach — Public Shadow Driving
o Public shadow driving for development can never produce these systems. You cannot overcome the time and costs to do so. And no one will allow for the public to be Guinea pigs as EACH company trains their machines to handle thousands of accident scenarios thousands of times each. (Technically that processes is already occurring. However, the AV makers are running benign scenarios now to avoid this. That is mostly due to the Elaine Herzberg tragedy.) Then there is the issue around handing over these systems to shadow or “safety” drivers” when they can’t handle something. That unnecessary process has caused the four deaths to date. What happens when the first child or family is killed by either of these processes? (More in my article below).
o Simulation is being used to support public shadow driving as well as test tracks
Opposite Approach — Aerospace/DoD/FAA Level Simulation-Full-Motion Simulators
o Simulation should replace the public version in 99.9% of the cases. This would be informed and validated by test tracks, driving with the driver under control and very limited and structure public shadow driving. A full motion simulator would be used where motion cues or the lack of them is critical to how the driver drives. For the critical core scenarios, a professional driver would be used. (That simulator would also test for motion sickness and passenger comfort.) Beyond this the industry needs to move to aerospace/DoD/FAA level simulation technology and safety practices. The reason for this is that the current simulation products in the industry have significant real-time and model fidelity issues, especially when vehicle, road or tire model’s performance curves are pushed or conditions degrade. (I would also like to mention the reliance on ML at this point. It is simply too poorly understood and does not function well enough to be the primary code driver. Proper systems engineering should drive most of the core systems especially safety. Meaning we need to box ML in to keep it from running amok or being wrong. AS such ML should be used to accelerate use case generation not be in command of it.)
External Data Dependence — Sensors
Current Approach — Reliance on Detailed Mapping
o Currently these systems rely on an outside sources for navigation and fixed/permanent object detection. Mostly detailed mapping. While some of it is understandable because sensors are not where they need to be, especially regarding degraded conditions, this practices is already a crutch. Along with GPS and V2X, which at 10hz is way too slow, these external information systems should rarely be primary dependencies for these vehicles. These vehicles must be able to navigate and detect objects on their own. Even when some of their internal sensors systems are degraded. There are simply too many ways the external sources could be compromised to allow for them to be critical in most cases.
Opposite Approach — Reliance on Primary and Secondary Internal Sensors
o These systems should use internal systems for primary and secondary navigation and object detection. At least to ensure these vehicles can get the occupants to safety until the other systems are restored. If these systems cannot prove they have the capability to rely on outside sources as the third option, they should not be allowed in the public domain. (I would like to mention Object Detection. I am not suggesting the current method flip. Just that these systems are nowhere near good enough yet to be on the road — https://www.zmescience.com/science/neural-networks-image-recognition-26246243/)
Engineering Process (Test Scenario Generation)
Current Approach — Agile — Bottoms Up
o Since most of the personnel creating these systems come from IT or gaming they use an engineering approach they are familiar with. That being an Agile approach. This approach starts from the bottom up and often with just one part of the system. As engineers learn more they add more. In massive complex integrated systems like what we have here Agile will waste so much time no one will ever finish and go bankrupt trying. That is for several reasons. Parallel engineering and integration of most systems is not done, nor are the hardest use cases tackled in Agile until late in the process. This will cause massive rework and wasted time. Often taking on the hardest parts first or as early as possible results in finding issues that change all of most of the engineering.
Opposite Approach — Aerospace/DoD Systems Engineering — Top Down
o While parts of an Agile approach could be kept it must be largely supplanted by a top down systems engineering approach. (This is not the “waterfall” IT uses. Commercial IT has never used most best engineering practices. They are CMMI Level 2 at best. With A-SPICE etc the automotive industry is much better but nowhere near aerospace or DoD.
Openness
Current Approach –Test (Scenario) Information Not Released — Supplanted by Useless and Misleading Information
o Right now, no one shares any test data with the public and there are no sanctioned third-party validators. All we get is miles and disengagement data which either means little or is severely misleading. All that matters is what scenarios should be learned and which were.
Opposite Approach — Public Release of Testing (Scenario) Data
o All test data should be made public before these systems enter any domain humans are present. That would mean test tracks or the public. (In the limited cases test track or public data is needed to inform and validate the sim or it can do nothing those situations would be structured accordingly.) And these tests should be validated by a reputable third party.
Regulation
Current Approach — Almost none — Voluntary — Not enforced or Not Applicable
o Governments are largely hands off here with few direct and specific regulations. Yes, the EU is a bit tighter and we have the FMCSA in the states which has been used by NHTSA to stop the EasyMile development school shuttle in Florida. However, the FMCSA does not cover the private domain and NHTSA/NTSB have yet to realize or do anything about handover/L3 and public shadow driving for development beyond that school shuttle. (Which does beg the question — why are they only sparring our children in school shuttles not every other vehicle type — in or around them?) The reason for this is that supposedly anything prescriptive the government does will hinder progress. Primarily by telling how industry should do something. If you look back at history this is nonsense and never works. All voluntary adherence, frameworks or guidance do, versus detailed requirements, is provide legal air cover for the lowest performance. The wretched cybersecurity capabilities of pretty much every entity on the planet and the current capabilities of our airline industry demonstrate this in both extremes. As for being prescriptive of how technology should do something that is a red herring. No one has suggested anything other than WHAT should be done.
Opposite Approach — Detailed Government Performance Regulation and Oversight
o What we need here, in the model of the airline industry, FAA and its 6.4 sigma safety performance, is for the government to dictate what good is. If this were done the industry would actually accelerate its pace to VIABLE products. We would now have definitive exist criteria that would ensure a new and level playing field. And most of the hype would be eradicated. (For those who say this area is far more complex than air travel, which is largely designed to be non-complex, you are correct. That is an argument for doing this rather than not. And yes, this is much harder. If you can’t do your due diligence to a testable world class level in an area that involves human life you should try another occupation.)
Other relevant articles
End Public Shadow Driving-SAE Autonomous Vehicle Engineering Magazine
https://www.nxtbook.com/nxtbooks/sae/ave_201901/index.php
Impediments to Creating an Autonomous Vehicle
https://medium.com/predict/impediments-to-creating-an-autonomous-vehicle-d3cfee299749
