It’s always good to counter Google’s “never an accident that was our fault” with “we switch to manual to avoid having an accident that was our fault.” But your conclusion, while welcome, is mostly based on a few hours of nanny driving. The falsehood is that 95% of the work done part. Sort of like a screw and bolt with different thread patterns being forced together, those last few turns are never going to happen, and worse, those parts are now locked together.
The correct path for self driving cars, at some later point in humanities future, won’t be with smart cars at all. It will be with the same dumb (overly complicated?) cars we drive today, driven by androids. For all of the futuristic notions of autonomous cars, they are no smarter than a chess program (that doesn’t know it is playing chess). Other area’s of AI research, unrelated to driving, will be the key to self-thinking machines that can easily replicate the day to day human activities we currently take for granted.