Linda Baker, of FreightWaves, contacted me by email to ask me about the hacks carried out by Keen Security Lab, a security laboratory belonging to China’s tech giant Tencent, on Tesla vehicle driving assistance systems, which included tricking the windscreen wiper control system and lane change control. On April 3, FreightWaves published an article titled “Researchers trick Tesla Autopilot using stickers on the road” (pdf), quoting me.
The hacks in question, which included showing a series of images to the front camera of the Tesla which the windshield wiper mechanism identified as rain or, more dangerously, placing three small white stickers on a road, which ticked the vehicle into changing lanes, may lead Tesla to consider how to protect its vehicles against such eventualities, but as such, they do not bring into question the safety of the company’s vehicles, nor should they qualify for the company’s bug bounty program, which is the reason why they have become news.
No system can be protected against every eventuality and limits have to be set. As I commented to Linda, putting stickers on a road to mislead a vehicle’s sensors is more like setting a booby trap or something Wile E. Coyote would try than something that could happen in under normal circumstances. That said, if it is possible to do so, then further investigation is justified. The hacks do not in any way question the safety of autonomous driving; no matter how hard Tesla or any other company tries, it is not possible to protect a system against every idea a safety expert might come up with, and the limit, in my opinion is modifying the road environment.
Below, the full text of the questions and answers of my conversation with Linda:
P. How reputable is Tencent?
R. Tencent is a huge Chinese multinational investment holding conglomerate founded in 1998, one of the largest corporations in China. Tencent is the world’s largest gaming company, one of the world’s most valuable technology conglomerates, one of the world’s largest social media companies (the Chines social network and portal QQ among others), and one of the world’s largest venture capital firms and investment corporations. Its services include social networks, music, web portals, e-commerce, mobile games, internet services, payment systems, smartphones, and multiplayer online games, which are all among the world’s biggest and most successful in their categories. Keen Security Lab is one of the important constituent parts of Tencent Security, and focuses on the cutting-edge security research of mainstream PC/Mobile operating systems, applications, cloud computing technologies, IOT smart devices, etc.
P. How significant are Tencent’s findings about being able to commandeer the Tesla steering system and confusing the lane identification system?
R. Changes and alterations to the physical environment are generally considered outside the scope of attacks against self-driving systems. The designers of a system cannot anticipate all the possible artificial changes that could be made to the environment in order to trigger a certain behavior.
P. Tesla says this report is not eligible for their bug bounty program. Are the bug bounty programs sufficient to detect software bugs, especially in self-driving vehicles?
R. Bug bounty programs are highly recommendable to introduce external takes on problems and issues. Paying people to troubleshoot computer systems is now considered an increasingly important part of organizations’ security, and they are important in terms of both effectiveness and reputation. However, the issue discovered by Tencent Keen Security Lab is not a bug, not a problem with the software or a limitation of the hardware, but the purposefully modification of a road, more akin to a booby trap. The findings are interesting and might merit some actions from Tesla in order to improve the vehicle’s behavior under certain circumstances, but changes and alterations to the physical environment are generally considered outside the scope of attacks against self-driving systems because it would be extremely difficult to set a proper limit for this type of interventions.
(En español, aquí)