Can a Neural Network spot Tyler Durden? Using Dlib and OpenCV for Face Detection and Recognition
You heard it right! Tyler Durden! The protagonist and the antagonist of the movie Fight Club. We are gonna see if a Neural Network can spot the Tyler Durden flashes in the movie!
[SPOILERS] The big twist of the movie was that Tyler isn’t real. He was a figment of The Narrator's imagination. What is more interesting is that the movie is full of hidden clues, and one of them is that Tyler Durden is flashed 4 times even before he is introduced in the movie! I never noticed it while watching the movie! [SPOILERS]
Today, we are going to build a Face Recognition model to see if it can spot Tyler in these flashes.
Face Detection
The first step to recognizing faces is to detect a face first! There are many ways to do it using OpenCV and Dlib.
- Using OpenCV’s dnn module.
OpenCV 3.3 officially released the “Deep Neural Networks” (dnn) module that supports Caffe, TensorFlow, Torch and PyTorch Deep Learning frameworks. OpenCV doesn’t support training any Deep Learning networks (probably because it isn’t intended to be a tool for training). But, it does allow you to use pre-trained models from any of the above supported frameworks. While the ‘dnn’ module offers support for a variety of tasks, we are interested in detecting faces.
For a particular case, let’s take a pre-trained Caffe model. The dnn module has a ‘readNetFromCaffe’ method that allows you to load a Caffe model for your use. The ‘readNetFromCaffe’ method takes 2 arguments- pre-trained Caffe model and Caffe prototxt. Then you pass the input image to get the bounding box for the detected faces. You can find the code for this here.
2. Using OpenCV’s Cascade Classifier.
OpenCV has a “CascadeClassifier” class that allows you to load the pre-trained Haar Cascades to detect faces, and also LBP Cascades to detect faces. The trade-off between the two is that- Haar Cascades are more accurate than LBP Cascades, but LBP Cascades are significantly faster than Haar Cascades. So, if you want a faster detection while compromising on the accuracy of detection, you should choose LBP Cascades. If you want a more accurate detection while compromising on the speed of detection, you should choose Haar Cascades. You can find the XML files for the pre-trained Haar and LBP Cascades here and here respectively. You can find the code to use both here.
3. Using Dlib’s Face Detector
Dlib has a “get_frontal_face_detector” method that allows you to detect faces. It is built using Histogram of Oriented Gradients feature combined with a linear classifier, an image pyramid, and sliding window detection scheme. You can find the code to detect faces using Dlib’s Face Dectector here.
Below are the results of using all the above approaches to detect face:
Face Recognition
Now that we have figured out how to detect faces in the image, it is time to use this to recognise the different faces!
There are 2 ways to perform face recognition:
- Using OpenCV’s Face Recogniser
OpenCV has three face recognisers- EigenFaceRecognizer, FisherFaceRecognizer and LBPHFisherFaceRecognizer. They are all part of the face module.
The EigneFaceRecogniser is based on Eigenfaces. Eigenfaces are a set of eigen vectors and are formed by performing Principal Component Analysis on your training data. The Eigenfaces are essentially the basis set for all the images.
The FisherFaceRecogniser is based on Fisherfaces. Fisherfaces are an improved version of Eigenfaces. The limitation of Eigenfaces are that it doesn’t really consider the difference in faces of different people. Since it looks at all the faces in your training data and computes the principal components, it doesn’t try to distinguish one person’s face from another person’s face. So, it creates features that represent all the faces in your training data as a whole. The Fisherface overcomes this by computing the principal components that considers the difference in different people’s faces. Essentially, the principal components differentiates one person’s face from the others.
The LBPHFisherFaceRecognizer is based on Local Binary Patterns Histograms. The limitations with Eigenfaces and Fisherfaces are that they are prone to the light conditions of the image. This means that they consider the face of a person taken in different light conditions to be different. LBPH overcomes this. It basically looks at local structure of the image by comparing each pixel to the neighboring pixels.
You can find the code for this here and here.
2. Using dlib’s face recognition embeddings
Dlib allows you to load a Deep Neural Network based Face Recognition model that is trained using a technique called Deep Metric Learning. Given an image of a face, this model outputs a feature vector of length 128. During training, the model is fed 3 images at a time- query image, positive image and negative image. This way, the model learns to output features that are almost same for a given person’s face under different transformations. This feature vector is used to quantify a face. For your training set, you compute the face features vectors for all the images. Then during testing, you compute the face features vector of the test image and compare it with the face features vectors of the train set. Based on the comparison (using a distance function), you find the closest face features vectors in the training set and assign the corresponding label to the test image. You can find the code this here and here.
Below are the results of model that recognises “Tyler” and “Not_Tyler”:
Now that we have built the Face Recognition model, let’s see how it does in spotting the Tyler Durden Flashes:
- Can a Neural Network spot Tyler Durden? #1
2. Can a Neural Network spot Tyler Durden? #2
3. Can a Neural Network spot Tyler Durden? #3
4. Can a Neural Network spot Tyler Durden? #4
This is so cool! When I first watched the movie, I never noticed these flashes. And now we have built a Face Recognition model that can spot these flashes!
You can find all the code in this repository.
Credits
The credits go to the creators, maintainers and contributors of OpenCV and Dlib, without which this wouldn’t have been possible. Kudos for their work! And obviously to David Fincher and the Fight Club team! Without the Tyler Durden flashes, this post probably wouldn’t have been possible.
