The problem with autonomous driving
Why we might not see self-driving cars in the near future
Introduction
Since some break-throughs in the early 2010s AI and machine learning have become an enormous hype. Finance, health, and education — everyone races to apply AI in their respective domain. Especially in the transportation area it has raised big hopes. In the last 10 years a lot of car manufacturers and tech companies have launched projects with the ultimate goal of achieving autonomous driving. Adaptive cruise control and self-parking are already almost standard features in high-end cars. Tesla takes it one step further and offers lane-centering and autosteer amongst other things. And watching cars of Google subsidiary Waymo drive through the suburbs of Phoenix is an almost eerie experience. So, driverless cars for the masses should be right around the corner, right?
Attack of the pixels
Take a look at the two pictures below: what do they both have in common?
They are both deceptions, made to fool us! The right one is pretty obvious: eight points with some white lines inside that put together in this way create the illusion of a 3D cube. But what about the left one? It clearly shows a panda, right? Put into a neural net specialized on image recognition this will tell you that this picture indeed shows a “gibbon”.
This example is taken from the now-infamous paper titled “Explaining and harnessing adversarial examples” written by Goodfellow et al. Here they describe how to make tiny changes that are not noticeable by humans to seriously distort the performance of image recognizers. By adding very small pixel perturbation to the image it becomes completely different — according to the neural net!
Researchers from the Kyushu University in Japan took this even one step further: by changing a single pixel in a 256 px ×256 px (= 65536 pixels total) they were able to fool a neural net. Take a look at image 3. The word in black is the class assigned to the image when fed the unmanipulated images, the word in blue is the result of the changed image. The predictions of the manipulated images differs massivly from the ones of the original images. The changes are so small to the human eye that they were marked with a red circle to make them better visible.
How can this happen? Neural networks can generally be described as black-box models: we give them an input and they produce an output but what happens in-between is not really sure. So, it is still not sure where these massive accidents can happen, even though there are several papers describing how to generate adversarial images. In his original paper, Goodfellow notes that
Adversarial examples can be explained as a property of high-dimensional dot products.They are a result of models being too linear, rather than too nonlinear.
Even though this is interesting it seems more like a gem that some AI researchers stumbled upon. So, should we care about adversarial images? Yes, definitely!
Autonomous driving
Already back in 2017 researchers from the Computer Vision Foundation (CVF) experimented on how to exploit adversarial examples in the real world. They came up with black and white patches to directly apply on signs (image 4). They fed this into a neural network which in turn predicted this to be a “Speed limit 45 mph” sign. Note: they didn’t use the visual system of a vehicle but used several standard model architectures in computer vision. In contrast to the images from above the changes made to the sign are relatively small but not invisible for humans. The complete failure of the model to classify the sign is nonetheless interesting.
Even more alarming are the findings of employees of McAfee Labs from February 2020. They managed to fool the camera system of a Tesla model. The original and unaltered version of the road sign was classified correctly with high confidence (image 5). Applying some changes to the sign fooled the classification system into seeing a “speed limit 45” sign. This clearly demonstrates that “hacking” a machine learning model used in the real world is definitely possible.
What can we learn from all this?
A paper published by IBM research states that
The quest of understanding this phenomenon has turned into an arms race
of attack and defense strategies. Along with the drive for efficient
and better attacks, there is a parallel hunt for effective defenses
which can guard against them.
So what we have seen in web security for decades now, the constant race between ant-virus software and new malware might already be an issue in AI security, too. It is important to acknowledge that the failure of machine learning system can not only be due to shortcomings of the model itself but also because of malicious attacks of future “AI hackers”. Several papers suggested that generating and training models on adversarial images can improve the robustness of said models. So further research in this area is absolutely necessary.
