The Era of Autonomous Cars: Focus on Pedestrian Detection with Deep Learning

Nowadays, we see a lot of potentially disrupting technologies. Autonomous vehicles are one of the most in vogue today!

Lots of amazing things can come out of this revolution: fuel economy, reduction of cars on the road, decline of accident and death rates, stress reduction, reduction of parking space and a real benefit for the environment.

Governments are already thinking about it. The National Highway Traffic Safety Administration in the US provides an official self-driving car classification. Europeans have also started modifying the Vienna Convention on Road Traffic and the Geneva Convention on Road Traffic in order to be able to adapt to this new technology. So, brace yourselves, the wheels are in motion (yes, pun intended).

Even though researchers have been studying the topic for years, it’s thanks mainly to Google’s or Tesla’s investments that autonomous cars are finally gaining traction today

So, how does it all work? From a more technical point of view, autonomous driving touches many challenges in different fields, especially computer vision. It requires improvement in object detection, image classification, environment reconstruction, motion estimation, tracking, scene understanding and many others.

One of the most talked about is pedestrian detection, so let’s explore this from a more technical point of view.

To welcome vehicles without drivers into our lives, we must be sure that they won’t be a threat and run us over.

Let’s look at where object detection is since pedestrian detection is a canonical sub-problem.

The most trending methods to reach optimal performance in pedestrian detection are Integral Channel Feature detector (ICF)(from Fast Feature Pyramids for Object Detection and Integral Channel Features) with many variants and the use of a pre-trained model using a convolutional neural network (Taking a Deeper Look at Pedestrians and Pedestrian Detection aided by Deep Learning Semantic Tasks).

Research papers show impressive gains in performance, but despite this, a human still widely outperforms state of the art AI detectors.

Detectors can make two types of errors. False positives (it detects something where there is nothing to detect) and false negatives (it doesn’t detect where it should)

The most common errors for detectors are:

• detecting tree leaves or traffic lights in background as pedestrian
• detecting the same person twice
• not detecting small persons
• not detecting cyclists

The under-representation of cyclists and side-view persons in the training sets used to train the detection algorithms could explain why partially occluded people and cyclists are not correctly detected most of the time.

Looking at the datasets, we see that small people are commonly saturated and blurry, which might explain weak detection. But when this factor is studied (How far are we from solving pedestrian detection?), no correlation between low detection score and low contrast is found. This also applies to blurred cases. There, the issue must be the smaller number of pixels.

Other approaches like (A Time Delay Neural Network Algorithm for Real-Time Pedestrian Recognition use a real-time algorithm for the detection and tracking of image regions that possibly contain pedestrians. It then uses a classification algorithm based on the typical motion patterns of a pedestrian’s legs. These approaches assume the visibility of the legs and works for walking people only!

Shape-based techniques, analysing the shape of objects, here the human morphological aspects like torsos, heads and limbs, allow the recognition of both moving and stationary pedestrians but the wide variety in pedestrian appearance sometimes makes this method inaccurate.

Approaches like (Shape-based Pedestrian Detection) use the morphological characteristics and the vertical symmetry of the human shape. Combined with a refinement of the stereoscopic techniques and use of the temporal correlation between subsequent frames, it appears to be robust for the differentiation of pedestrians from other objects such motorbikes, trees and traffic lights.

There are many things out there working on keeping us alive in a world of autonomous cars! While extraordinary achievements have been made, there is still work to do to get acceptable performance and see autonomous cars among us. But as with the other fields in the fields of AI, autonomous machines and computer vision are continuously improving, I feel we’re almost there! Get ready to leave your driver’s license at home. :)

If you want to read more on the topic, I recommend this page, as well as all sources linked in this article.

Originally published at Recast.AI Blog.