How to spend 1,000x more money on self-driving and still lose

Consumer Reports recently released a review of the current driving assistance monitoring systems. This is for the Adaptive Cruise Control (ACC) and Lane Keeping Assistance (LKA) features in new cars.

Here are the results:

Shameless plug: The founder of comma.ai, George Hotz, did a great interview with Lex Fridman that I transcribed on Deep Chats, a platform I built.

Comma Two came in first place, which is surprising considering it’s basically a cell phone mounted to your car that hacks your cars camera and control functions to drive.

Naturally, Consumer Reports needed this caveat:

Although Consumer Reports does not endorse after-market modifications to all consumers, we feel that it is important to include the test results in this report to the industry. The direct comparison of this system to the other OEM systems will hopefully provide insight on this alternative approach and highlight the areas across the industry that have room for improvement

I see the fact that its an after-market mod a feature not a bug. It makes sense to separate the hardware from the software. And you can have a lot more flexibility on the software side and not be tied to the annual release cycle of most auto-makers.

Let’s put these scores against how much each company spent on R&D. I saw this tweet from a former comma.ai intern caught my eye.

I didn’t put together the numbers, but the comma.ai number is ballpark correct since they raised 8.1 million since 2015. They also sold a few thousand units at 1k and probably have at least a million in the bank, so $1.6 million is a reasonable estimate for a 5 year old company with about a dozen employees.

Tesla spent $1.3 billion in R&D in 2019, probably much of it on self driving. That’s off from 2.9 billion from the table above. I assume the author of the tweet included all expenses, not just those tied to self-driving. Same is true for the rest.

It’s hard to tease out, but its conceivable that the large auto-makers are spending at least $1 billion every year in this industry. Audi is planning on spending $16 billion in the next 3 years on self-driving tech and electrification.

So how is comma.ai able to do so much with so little?

What you need to build a self driving car

1. Something to control the car (e.g. steering and acceleration)

Most new cars already have some form of control interface (e.g. auto-breaking, collision detection, power steering). Car companies already have people working in these fields and should be able to extend their scope to coordinate with the self-driving team.

comma.ai skirted both of those problems since they basically hijacked the existing car’s monitors and cameras and used that as a starting point. Their hardware is a cellphone and a few connector cables. And they leverage the open source and hobbyist community in to adapt new cars.

2. Something to take in observations (e.g. camera and/or lidar)

Same as with 1, most new cars already have cameras and a system that uses the camera as an input to some behavior (e.g. auto-braking)

3. Decision network (e.g. trained neural network)

This one is harder since its still an unsolved problem. But it helps to narrow down the problem. And, to be fair, most of the auto companies do that to a certain extent. They do “lane assist”, just keep you in your lane and brake if you’re approaching something too quickly. Tesla is the only one that is more ambitious: integrating maps, navigation and parking.

4. Lots of data

A successful driving model is very much dependent on your edge cases. And to get those edge cases you need a lot of data. Bonus points if you don’t have to pay for it.

The big advantage Tesla and comma.ai over the others is that they have millions of miles of test data to train their networks. Tesla likely has even more including all the relevant censor information, while comma.ai only has a single camera video for the most part. That’s why its important to get a minimum viable product out there to iterate on.

What you don’t need to build a self driving car

1. Tens of billions of dollars

Okay, that’s not fair since no one built a commercially available level 5 autonomous vehicle. So maybe it does take that much money and maybe Audi will make that leap in 3 years after their $16 billion spend.

But what does tens of billions of dollars buy you? A lot of engineers, sure, but when building out a decision model, what’s the marginal contribution of the second engineer? How about the tenth? The hundredth? At the end of the day, its still a technical problem and mythical man month principle applies. You do need a lot of money for compute resources, but that’s in the millions, not billions.

For the decision model, you need a few dozen talented engineers, millions of dollars in compute and a lot of real world driving data. Then hand it off to production to incorporate it into a vehicle. The model can and should be developed as independently as possible from the other functions of the vehicle.

2. A car built from the ground up

Which is more plausible, self driving electric vehicles built from ethically sourced recycled parts or a cell phone that can hook into your car and keep you in your lane? Separation of concerns principle applies, as does bike shedding

3. AI Ethicist

Question: if your self driving car is about to hit an old lady which it can only avoid if it hits a young child, what should it do? Imagine getting this question on a driving exam.

There is no place for an explicitly programmed ethics module in a self-driving model, nor have I ever heard a technical approach to incorporate such a system. As with any profession with a large financial backing, the AI industry is not free from snake-oil salesmen. AI ethics in self-driving is for magazine articles not for engineers.

4. Infrastructure for self driving cars

The worst idea in the self-driving space is that you need special roads or self driving cars that communicate with each other. Building an interconnected complex system leads to a fragility and systematic risk. You should prefer decentralization and unsystematic risk if possible.

The fact that each car is independent and uncorrelated is why, as a society, we accept such high mortality rates. Building complicated networks that talk to each other and act on those results would lead to disastrous results. Engineering complexity has to be justified, and the efficiency to be gained is not worth the added overhead and systematic risk.

Who’s going to win?

Probably Tesla. They seem to be doing everything right, for the most part. They have a the product, data, method of deliver, money and engineering culture. Once that nut is cracked, others will come along in the market. Kind of like when someone breaks the 4 minute mile others follow shortly after. And there it makes sense for a newcomer or even comma.ai to be able to compete seriously through a mostly software approach. But personally, I’m not holding out hope for Audi’s $16 billion investment.