Anti-Spoofing Mechanisms in Face Recognition Based on DNN

Published in

The Startup

6 min readMar 14, 2018

Imagine being able to unlock your phone just by using your face, no fingerprint scanning or touching required. It would just work, automagically and flawlessly without any user intervention. Wouldn’t that be great?

Well, guess what…someone already made this happen. It’s called the iPhone X and you might just be using it already. But what’s even better: the potential for using face recognition for user authentication is much bigger than this! In the not-so-distant future, we’ll hopefully be able to rent a car, sign legal documents and do everything else just by showing our unique facial features.

We’re already starting to see it with certain services requiring ID verification (like banking and other types of transactional systems). In this case, the provided legal data is cross-checked with data on the ID and face image on a document is compared to the owner’s face. However, like most new technologies it introduces new possibilities of a breach. And one of the most popular ways to deceive a face recognition mechanism is a ‘face spoof’ attack.

A spoofing attack is an attempt to acquire someone else’s privileges or access rights by using a photo, video or a different substitute for an authorized person’s face. Some examples of attacks that come to mind:

Print attack: The attacker uses someone’s photo. The image is printed or displayed on a digital device.
Replay/video attack: A more sophisticated way to trick the system, which usually requires a looped video of a victim’s face. This approach ensures behavior and facial movements to look more ‘natural’ compared to holding someone’s photo.
3D mask attack: During this type of attack, a mask is used as the tool of choice for spoofing. It’s an even more sophisticated attack than playing a face video. In addition to natural facial movements, it enables ways to deceive some extra layers of protection such as depth sensors.

Spoofing Detection Approach

Some form of security should become standard in all facial recognition based systems. There are many different approaches to tackle this challenge. The most popular anti-spoofing state-of-the-art solutions include:

Face liveness detection: A mechanism based on an analysis of how ‘alive’ a test face is. This is usually done by checking eye movement, such as blinking and face motion.
Contextual information techniques: By investigating the surroundings of the image, we can try detecting if there was a digital device or photo paper in the scanned area.
Texture analysis: Here small texture parts of the input image are probed in order to find patterns in spoofed and real images.
User interaction: By asking the user to perform an action (turning head left/right, smiling, blinking eyes) the machine can detect if the action has been performed in a natural way which resembles human interaction.

And of course, we can’t ignore the elephant in the room, FaceID on the iPhone X. In the latest hardware iteration, Apple has introduced advanced depth-mapping and 3D-sensing techniques which enable spoofing detection with unprecedented accuracy. However, as this high-end hardware will not be available on the majority of consumer devices in the near future, we think it makes sense to double-down on what’s possible with existing 2D cameras.

In fact, during our research and implementation, we found out that it’s possible to achieve an extremely high level of real-time spoofing detection with a medium-quality 2D camera. The secret? Using a Deep Learning solution with a custom neural network.

We validated our approach by cross-checking it with existing, documented approaches.

Cross-check 1: Image Quality Assessment

This solution is based on comparing the original image with an image processed with Gaussian filtering. Author of the paper [1] is proving that differences between fake images vary from real ones and it could be detected automatically. In order to do that, we’re extracting 14 popular image quality features such as Mean Squared Error, Average Difference or Total Edge/Corner Difference. The next step is to send them to the classifier in order to determine if it’s a ‘real’ face or ‘fake’ face.

Figure 1. IQA classifying process flow: converting the image to grayscale, using Gaussian filter, extracting 14 features from the difference between original and filtered image, passing features to the classifier.

Cross-check 2: Image Distortion Analysis

Four different features (specular reflection, blurriness, chromatic moment and color diversity) are sent for classification. The classifier is built with multiple models, each of them trained on a different type of spoofing attack vector.

Figure 2. IDA classifying process flow: extract 4 distortion features, pass them into a group of classifiers, pass results into classifier responsible for spoofing/not-spoofing decision

Final approach: Deep Neural Network model

This is based on a model built with CNN (Convolutional Neural Network, which is the most popular neural network in image analysis). A cropped image of a face is passed into the neural network and then processed through neural layers in order to classify it as real/fake.

Figure 3. DNN classifying process flow: pass face image into CNN

Training The System

All of the solutions presented above contain models which require supervised learning in order to return proper results. A training set was built from images of people standing in front of the camera or holding a device with their face displayed on it. All faces were cropped and divided into two groups: real and fake. The images were taken by a medium-resolution 8MP camera, a popular model used in industrial applications. The same training set was used for all three approaches.

Comparing Results

For performance measurements, we used simple precision, recall, and F1-score. The result of all three experiments is shown in the table below.

What’s Next?

The presented, state-of-the-art solution only works with 2D replay/video attacks. In order to increase resistance to more types of attacks, the DNN model could be tweaked by extending training data with paper-printed attack examples. Additionally, 3D spoofing attempts could be handled by additional sensors (for example depth).

Security is a constantly evolving matter since attackers keep finding new ways to breach the system as soon as new protection methods are being introduced. But we think our unique approach could already be applied to all processes involving automatic (or semi-automatic) KYC validation to decrease the number of fraudulent accounts, or at the very least reduce the amount of manual labor (final validation) required.

Curious to try it out yourself? If you’re an iPhone user you can download our facespoofing app and give it a go!

UPDATE 03/19: We are currently working on a cloud-based solution, we’ll update this blogpost as soon as we need beta testers. Stay tuned!

This post was written by Nicky Jackie Delphin & Artur Baćmaga, one of YND’s AI experts. With over 6 years of experience in Python, Artur works as an ML/Python Developer at YND. He leads AI-powered processes for projects such as Car Detection & SmartBar. In need of some brain power? Reach out to us via hello@ynd.co with your questions about ML/AI projects.

References:

¹ Face Anti-Spoofing Based on General Image Quality Assessment, Javier Galbally, Sébastien Marcel

²Face Spoof Detection with Image Distortion Analysis, Di Wen, Hu Han,Anil K. Jain

³Biometric Antispoofing Methods: A Survey in Face Recognition, Javier Galbally, Sébastien Marcel, Julian Fierrez