Why I Connected My Car To The Blockchain (And You Should Too)
TL;DR: enterprise-developed AI has a principal / agent problem. Until it is solved, proposed regulation of AI is really just wishful thinking because deployed AI systems cannot be audited transparently. I explain how I connected my Tesla's sensors to a public blockchain to solve this problem in AV.
My new car is fun
I'm part of a growing number of "early adopters", or "gullible idiots" depending on who you believe, who decided to buy a "Fully Self Driving" Tesla Model 3. I happily plonked down the additional 7000$ on a rainy day in October to participate in the ushering-in of a new era of personal-mobility freedom, thereby (hopefully) ingratiating myself to our future robot overlords.
My car was quickly delivered, to be picked up in a large tin shed near the local airport (Brussels, I live in Belgium).
The actual delivery of the car was so momentously anti-climactic and disorganized that it might warrant another article... Suffice to say that I left the tin shed mildly disgruntled, but with my motoring-enthusiast soul billowed by the winds of knuckle-whitening acceleration and the promise of effortless long-distance cruising.
I had spent months researching the car, it's equipment, the mechanics and electronics, and of course the capabilities of the "full self driving" AI-system. So I knew not to set the car to "AV mode" as soon as I exited the parking lot into Friday night commute traffic.
My family were delighted: the interior is far more luxurious than any car we have ever owned before, the entertainment system is stunning (nothing gets my kids more excited than the potential for Frozen 2 karaoke with _every_ _single_ _trip_), and of course the astounding smoothness and silence of the all-electric drive are fantastic… But I was ready to unleash the AV robot!
I decided to wait a little, however.
The incentive problem in enterprise Machine Learning
I have been thinking about artificial intelligence for a while now, using machine learning tools in my day-to-day work for the last few years. I started to incorporate ML tools into the 3D body scanning systems that I design with my company Treedy's. Our uses of AI are fairly innocuous. We use a variety of tools, mostly internally developed, to:
- improve the quality of point cloud data from 3D sensors (we have the state of the art in DNN de-noising of 3D data with an internally-developed method)
- infer people's body shape and dimensions underneath clothing (meaning you don't need to get undressed to get scanned, this is a system that we created from scratch working end-to-end with AI on 3D data)
- transfer measurement lines from a template human body to the people we scan so that we can extract relevant dimensions (we also have the state of the art in this, to be published soon)
- a bunch of smaller tasks like segmenting images and 3D point clouds to separate clothing, hair, accessories etc…
Treedy's mission is to develop technology to 3D scan people with minimal effort in order to extract precise body measurements that help people shop for clothing online in a more sustainable way.
In other words: if any of our AI systems are off, the worst that can happen is that you get a sweater that's a little tight!
Treedy's is very small team with internally-aligned goals and incentives, and there is no external government body that needs to provide oversight or make sure we are not screwing up when measuring someone's elbow diameter.
There's no need to establish who is responsible if our results are wrong… If our algorithms suck it will hurt our business, and maybe our customers' businesses, but that would be the end of the damage (they don't suck, we are awesome 😁).
But AI for cars is a very different animal
Artificial intelligence for autonomous vehicles, especially when developed in a large organization like an automotive OEM, is a complex problem of incentives.
Consider the simplified diagram below of how an organization working on a self-driving car might work:
Now let's assume that this company's over-arching mission statement is "safety first": the company's board of directors signed off on this statement, and the CEO is tasked with defending this vision in front of regulators and customers.
But what about this engineer?
This person is tasked with designing an AI system that is acting on a limited set of training data to get the best results in "self-driving". Sure, they have been made aware of the company's "safety first" motto, but they actually are being evaluated on objectives and goals set by their manager, and need to deliver very tangible results in a limited time frame.
In other words, this is a principal / agent game-theoretical problem: the principal's objective is "safety first", but the agent's objective is "deliver all of this stuff by date X", leaving some room for interpretation as to the exact sense and importance of the company's motto.
Now let's take a look at another scenario. The deployed AI system fails, and we need to find out what happened, because:
- a regulatory body needs to determine whether the company deploying the system acted lawfully both in designing the system and in deployment
- the company wants to determine whether the cause of the failure is something that can be rectified by changing an internal process
- the company wants de determine whether the cause of the failure can be traced back to an error from a supplier
- etc..
Any one of these scenarios could involve long and costly litigation and investigation, and if mishandled could mean the death of our "ACME AV company"…
Ok so let's investigate!
First off: who is charged with finding out where the failure originated?
For the sake of clarity in this example let's suppose it's an AV that failed in a country that has some basic legal infrastructure for dealing with AV testing and there is a government body that is charged with digging into the case, like the NTSB in the US.
In order to find out what happened the NTSB will have to get access to the data captured by the AV's sensors in the moments leading up to the failure event. Ideally, they will also need to have access to the deployed machine learning binaries in an executable format in order to reproduce the failure scenario.
Ok, so who might have access to the sensor data and AI binaries or code, to hand these over to investigators?
Depending on how the data is stored and can be recovered from the vehicle, in this hypothetical scenario any or all of these agents could have access to this data:
….and here we run into the principal / agent problem once again:
Principal: The CEO, who is defending the company's "safety first" mission statement and has faith in their employees, will urge all agents to provide transparent access to any data available to the NTSB.
Agents: All of the employees and subcontractors at different levels of the system have different incentives, which might be at odds with one another, and might push them to purposefully modify or destroy data that would be relevant to an investigation!
In the words of Charlie Munger :
"Show me the incentive and I will show you the outcome."
Eine minute bitte!
But wait a minute… something like this has already happened in the automotive world very recently with Volkswagen's "Dieselgate".
It's a very similar problem: a set of agents are tasked with the objective "make this car get better results in emissions tests", and decide to not adhere to the standards of ethical judgement that are expected (hopefully) by the principal.
In this case they deployed software on-board cars that detected when the car was being subjected to an emissions test and adjusted certain components accordingly to display favorable results in the test. Classy move 🙄
According to the most recent reports, the repercussions of "Dieselgate" have already cost VW upwards of 30B USD, with more to come… and that's without factoring in the cost in terms of image and trust lost by consumers.
Let's also not forget that this decision is literally killing people!
Back to AV 🤖
AV is a nascent industry in which most actors are tremendously over-valued and have no clear path to profitability (check out this great article by comma AI on the subject)… And a scandal like Dieselgate in which sensor data is falsified by an agent acting on their own role-specific incentives might not just take out a single company, but may have the potential to wipe out the entire industry.
“Hell is truth seen too late.” -Thomas Hobbes, Leviathan
A proposed solution: Using public blockchains as data authentication services
After thinking about this problem for a long time, a couple of friends and I set out to propose a solution with a company that we co-founded called Sensor Link.
We built a system that can act as a third-party verifiable registry of data authenticity. Essentially a way to remove any incentive to falsify or hide sensor- or inferential data by making such actions evident to third-party inspectors.
It works like this:
For each specific point in time, at a frequency to be determined based on the specifications of the AV system, a set of cryptographic hashes are collected.
These hashes are fingerprints of the data that is used as input to- as well as the data output by- the "self-driving" system. These fingerprints are uploaded in real-time to an API that we have developed, which archives the fingerprints, arranges them into a hash tree, calculates the "root hash" of the tree and commits this root hash to a public and immutable database (a well-decentralized blockchain in this case).
The Sensor Link database itself is immutable: if we make any changes to a single data point within a specific time frame it will change the root hash for that time frame… and by committing the root hash to an immutable, decentralized database we make sure that this is not possible without corrupting the entire database.
So now the Sensor Link database can be interrogated for a specific user at a specific time to collect all of the "authenticity fingerprints" of all data used on-board the AV.
Remember these guys, who have access to the data after a failure event?
Using the same (publicly documented) cryptographic algorithms that are implemented on the AV itself, the data collected from these agents can now be processed and the extracted cryptographic fingerprints can be compared to the records in the Sensor Link database… and if anything doesn't match, we can spot the bad actors in the system!
We believe that a record of "data authenticity" should be a requirement for AI-based systems operating in the public space, and that the "principals" of organizations developing such systems should think long and hard about implementing a solution to make sure that their objectives are not being undermined by poor decisions on the part of the agents executing their strategy.
…so I plugged my car into the blockchain!
A few months ago, I stumbled upon a lovely GitHub repo called "teslaUSB" ( the original repo is here, but there is more recent development on this excellent fork).
As the original author mentions in the title page:
You can configure a Raspberry Pi Zero W or Raspberry Pi 4 so that your Tesla thinks it’s a USB drive and will write dashcam footage to it. Since it’s a computer:
Scripts running on the Pi can automatically copy the clips to an archive server when you get home.
I set up a Raspberry Pi and started hacking away, and soon had it committing cryptographic fingerprints of the video files recorded by my Tesla's sensor array to the Ethereum blockchain via the Sensor Link API, in real-time with the Raspberry Po connected to my iPhone acting as a 4G gateway!
You can check out a certificate of data authenticity, generated by the API here… it includes the hash tree proof and a link to the transaction on Ethereum where you can find the root of the hash tree.
What does this mean?
In short: if the AV system in my car fails catastrophically I will have:
- A copy of the footage recorded by the sensors on a local storage device (for now this is limited to what Tesla records to the USB storage device)
- Provable authenticity and completeness of the data that I have stored via the Sensor Link database and API
… making it possible to ensure a transparent investigation!
What's next?
For Sensor Link: we are working on building a public version of our API for people and companies to experiment with the possibility of having provable data authenticity without the complexity and mess of physical "black boxes"… but for now if you’d like to know more please take a look at the Sensor Link technology sandbox at www.etherstamp.io !
If you are interested in setting up your own connection to the Sensor Link API please make sure to connect with us over at www.sensor.link, we would love to hear more!
For your own Tesla: more details, including how to connect your own Tesla to our API, will be published soon… watch this space!
Special thanks to: Elon Musk for making super cool stuff and ConsenSys for helping us to start Sensor Link in the first place! Also a big shout-out to Cimryan and Marcone over on GitHub without whom this would have taken much much more work to pull off, thanks guys!
To anyone at Tesla Belgium: any chance you guys have found my winter tires yet?
