Face Recognition with VGG-Face in Keras.
1. Get dataset/images of persons.
I have collected images of top 5 most powerful leaders in the world Donald Trump, Vladimir Putin, Xi Jinping, Angela Merkel, Narendara Modi. Train dataset contains 10 images of each person and also to check model working also included my images. Name this folder as “Images” .
2.Detect faces in image.
To get better predictions we first detect faces in image and use only faces for recognition. To do so we first detect faces in an image,for this we use ‘mmod_human_face_detector’ a cnn_face_detector which identifies faces in image and returns position of each face in image with finding rectangle bounding box as (left,top,right,bottom) positions. ‘dlib’ in python uses these weights and detect images,but dlib doesn’t provide this detector as in-built. We must download and provide to ‘dlib’ classes for extracting face. Download mmod_human_face_detector from here
As it is .bz2 file,extract it
$ wget http://dlib.net/files/mmod_human_face_detector.dat.bz2
$ bzip2 -dk mmod_human_face_detector.dat.bz2 3. Extract faces from images.
Extract face from images,crop face and store as image in separate folder with image name as person name.
We use images with only one face
Above snippet shows how to extract face from image and save them for recognition. Do the same for all images in train dataset and test dataset saving with person names as image names. Store each person cropped image in a separate folder like Ex: All ‘modi_*.jpg’ images are saved in ‘modi’ folder.
After extracting faces directory structures looks like:
Directory structure :
|Images /
| |-- (60 images)
|Images_crop /
| |--angelamerkel (10 images)
| |--jinping / (10 images)
| |--lakshminarayana / (10 images)
| |--modi / (10 images)
| |--putin / (10 images)
| |--trump / (10 images)
|Images_test /
| |-- .. / (18 images)
|Images_test_crop /
| |--angelamerkel / (3 images)
| |--jinping / (3 images)
| |--lakshminarayana / (3 imgaes)
| |--modi / (3 images)
| |--putin / (3 images)
|Face_Recognition.ipynb
|mmod_human_face_detector.dat4.Use VGG-face model to create embeddings for faces.
We create embeddings for each face/person which defines the person in numeric data. Pre-trained networks like DeepFace,OpenFace provides embeddings in less than 5 lines of code. But we use VGG_Face_net which trained on millions of images to recognize labelled faces in the wild (LFW). The original model takes an image in WildFace dataset on which VGG_face_net trained and classifies/recognize person in image. It ouputs 2622 embeddings for an image,we take this 2622 embeddings for each cropped_image for later classification of image. VGG_face_net weights are not available for tensorflow or keras models in official site, in this blog .mat weights are converted to .h5 file weights.
Donwnload .h5 weights file for VGG_Face_net here
‘https://drive.google.com/uc?id=1CPSeum3HpopfomUEK1gybeuIVoeJT_Eo’
As it stored in google drive,we can download to our local storage with python ‘gdown’ package.
$ gdown https://drive.google.com/ucid=1CPSeum3HpopfomUEK1gybeuIVoeJT_Eo We have .h5 vgg_face_net weights now and use it to build vgg_face_net model in keras/tensorflow.
5.Build vgg_face_architecture and get embeddings for faces.
To loal weights for model we must define model architecture. VGG_Face model in keras as
In the output layer they used softmax layer for recognising image in WildFaces dataset. We do only require embeddings which are output for last but one layer i.e,. Flatten() layer. So our model requires upto last Flatten() layer.
# Remove last Softmax layer and get model upto last flatten layer #with outputs 2622 units vgg_face=Model(inputs=model.layers[0].input,outputs=model.layers[-2].output)
In the above line we defined model upto Flatten() layer. Now we can feed any image to get embeddings which will be used to train our own classifier/recognizer.
6. Prepare train data and test data
Prepare train data and test data which contains embeddings as rows for each face and label as person name.
Earlier we stored each cropped face image in corresponding person folder, walk through each folder and in each folder for each image, load image from keras in-built function load_img() which is PIL image with
target_size=(224,224), since VGG_face_net expects image shape in (224,224) format. For each loaded image it is preprocessed into scale of [-1,1] and feed into vgg_face() model which outputs (1,2262) dimensional Tensor, it is converted into list and append to train and test data. Also,for each person we label that person in numeric number like
Ex: { 0: ‘modi’, 1: ‘trump’, 2: ‘angelamerkel’, 3: ‘jinping’, …. …. }
We got (x_train, y_train) and (x_test, y_test) as lists ,to use in keras models we first convert them to numpy arrays.
x_train=np.array(x_train)
y_train=np.array(y_train)
x_test=np.array(x_test)
y_test=np.array(y_test) 7. Train softmax classifier.
We end up with train data and test data with face embeddings and labels as person encodings. Now we train simple softmax classifier and save model to get predictions for unseen image face in dataset.
We trained softmax classifier to classify image, it takes face embeddings as input and outputs corresponding image number which is encode for person name.
8. Recognize faces.
Now we can recognize any face in image if we get embeddings for face with help of vgg_face model and feed into to classifier then get person name. With opencv draw rectangle box around face and write person name for each face in image.
In above snippet it takes image path and outputs recognized faces in image with rectangle box around face and person name.
My Image
