Detecting Cars and License Plates in Images Using Deep Learning
With deep learning, it is very easy to classify images (“Does it contain a dog, or a cat?”, “Is it a green, or red car?”). Yet, this is often not enough and the position of an object in an image or video stream is also required (“Is the dog in front of the door?”, “Where is the license plate of the car in front of the boom?”). Object detection has caught up quickly with image classification, allowing not just for accurate tracking of objects, but also for low compute power requirements that allow deployment of these algorithms on mobile devices.
At Eliiza, we are interested in driving such projects from vague idea, to proof of concept, to fine-tuned and usable product. Often, this last step can be a tremendous issue for companies, and we have the expertise to help. We have worked on a variety of such object detection issues like automatic utility meter reading, and more recently license plate detection and car identification for an automated parking payment system.
The Modelling
To get this last project off the ground, we curated the necessary training data of cars and license plates to train the algorithm effectively. For an acceptable model several thousand examples per object category are necessary, including both labels and bounding boxes. These requirements can quickly push a project to the level of millions of images. Utilising transfer learning, that is re-training a pre-trained model, can make a smaller amount of data in the area of hundreds of images per category sufficient for a good model.
For the object detection we chose the YOLO (“you only look once”) loss, which allows for a good combination of model accuracy and image processing speed.
Iterating over different model architectures and datasets, we arrived at a lean and fast model that can run in real-time speed on smartphones and recent embedded devices.
As a model is of use only if it can be put to work, we deployed a Docker container on an AWS Lightsail virtual private server under the URL object-detect.eliiza.ai/. In this way, a docker container can be deployed in under half an hour, ready for https queries. The container runs a Streamlit dashboard allowing one to upload one’s own images, or provide the URL of one’s images to be processed by our model. Moreover, the detection threshold can be modified with a slider, to test sensitivity to the detected car or license plate.
Ethical AI and Results
Computer vision is a field that has recently seen a large outflux of extremely high calibre researchers due to ethical concerns. At Eliiza, it is part of our core tenets to pursue ethical AI. Thus, we also trained this system to detect any and all humans, to automatically mask them before any data is to be passed on for secondary analytics/human review.
The above video shows the success, and one current weakness of our model. Cars, license plates, humans (and also the not shown bicycles, lorries, motorbikes, …) are accurately identified and located in each image, with high fidelity. Yet, they do need to be above a certain threshold size to be accurately recognised, which is a consequence of our model being slim enough to run on an embedded device or a smartphone in real time. Using hardware acceleration, the minimum amount of pixels necessary to cross the object detection threshold can be greatly reduced, while retaining or even improving the model’s speed and detection accuracy.
Caveat: license plates and humans are covered up with rectangles for privacy reasons. The full model also returns the read-out license plate letters and digits.
Opportunities
The possibilities for such a technology are vast, especially on live videos. One could for example use it to automatically detect groceries (is it a Braeburn, a Fuji, a Granny Smith, Jonagold, … apple?) which are hard to identify for a customer, thus speeding up the self-checkout process, reducing queues and increasing customer satisfaction. As an added bonus, it will also allow retailers to prevent mistaken sale of goods under the wrong label (buying an avocado, accidentally clicking on the apple symbol right next to it, adding a cheaper price to the total), and more accurate stock levels.
Object detection will also be able to show its strengths during the current ongoing covid-19 crisis, think e.g. face and mask, or automated face and temperature detection at airports.
