Advanced Face Recognition System
Design and implement a comprehensive advanced face recognition system
Introduction
Facial recognition technology has been making headlines in recent years due to its increasing prevalence and potential applications in various industries. From unlocking smartphones and accessing secure areas to identifying suspects in criminal investigations, facial recognition technology has become an important tool in our modern society.
This article aims to guide readers through the process of designing and implementing a face recognition system from scratch. In a subsequent article, we will explore the development of an advanced real-life application of facial recognition technology, such as a door lock system based on biometric features or a facial recognition software to aid law enforcement authorities in identifying perpetrators.
The Pipeline
When we start to design any AI system, we start to design the pipeline which generally refers to the sequence of steps or processes involved in building and deploying an artificial intelligence model or system. In the context of machine learning, the AI pipeline typically involves several stages, including data collection, data reprocessing and cleaning, feature engineering, model training and validation, and deployment.
To better understand each step involved in our face recognition system, we will divide the pipeline into sub-pipelines. Then we will combine these sub-pipelines to create a comprehensive pipeline that describes the entire system.
1.0 Data Collection
Data collection is a significant challenge in the field of AI because having enough high-quality data is essential for building an effective AI system. For example, ChatGPT, a popular AI system, relies heavily on the quality and quantity of data used for its training. Whenever a user asks ChatGPT a question, the system references its training data to provide an answer. Therefore, if the training data is insufficient or of poor quality, the system is likely to generate unsatisfactory results.
I have experimented with several approaches to increase the accuracy of a face recognition model, and the most effective one was to use a full face scan of the user’s face as a video. Then divided this face scan video into a fixed number of images and used them for training the model. Below is an example of a Python code for dividing video to a fixed number of images.
import cv2
import os
# Input Path
video_path = '/home/jawabreh/Desktop/face_scan.MOV'
# Output Path
output_dir = '/home/jawabreh/Desktop/face-recognition/training/person_1'
# Create the output directory if it doesn't exist
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# Create a VideoCapture object to read the input video
cap = cv2.VideoCapture(video_path)
# Get the total number of frames in the video
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
# Calculate the frame interval to capture for 150 images
frame_interval = total_frames // 1000 # change this number according to your needs
# Set the initial frame counter to 0
frame_counter = 0
while cap.isOpened():
# Read a frame from the video
ret, frame = cap.read()
if not ret:
break
# Check if this is the frame to capture
if frame_counter % frame_interval == 0 and frame_counter // frame_interval < 1000:
# Save the frame as a JPEG image
output_path = os.path.join(output_dir, f'{frame_counter//frame_interval + 1:03}.jpg')
cv2.imwrite(output_path, frame)
# Increment the frame counter
frame_counter += 1
if frame_counter >= total_frames:
break
# Release the video capture object
cap.release()
print("\n\nDONE\n\n")What if the system is not intended for use by law enforcement authorities, and I want to gather data of celebrities or my friends to train the model? While we can certainly do so, we cannot expect to achieve the same level of accuracy. Do we need to manually search for and download each celebrity face image from the internet? Absolutely not. Here is a Python script that can retrieve data from the internet based on a specified keyword.
from simple_image_download import simple_image_download as simp
# from simp library call simple_image_download function
response = simp.simple_image_download
# the keywords that will be used to find pics, and each key work will create a different file
keywords = ["George Wassouf", "Donald Trump", "Selena Gomez"]
# for loop on the keywords
# (kw, 1000) means 300 sample of each keyword
for kw in keywords:
response().download(kw, 1000) If you decide to use the aforementioned Python script to collect training data from the internet, it is important to note that after collecting the data, you will need to manually filter it. This is necessary because the collected data may include unrelated images.
Alternatively, you can utilize pre-built datasets that have already been filtered and cleaned. you can use Kaggle platform to find pre-built datasets, here is an example of a pre-built dataset with many face samples of some celebrates.
https://www.kaggle.com/datasets/alihasnainch/celebrates-image-dataset
2.0 Feature Extraction
Facial recognition training is a crucial step in developing an accurate and effective facial recognition system. The training process involves feeding a large amount of facial data into the system, so it can learn to recognize different facial features and distinguish between individuals. The data can come from a variety of sources, including photos, videos, and 3D scans.
In any face recognition system, there are three main steps: face detection, feature extraction, classification and identification. During the training process, we focus on two of these main steps: face detection and face feature extraction.
The above figure illustrates the face recognition training process, first we use the training data that we already prepared it, we will have a main file named training inside this folder we will have a separated folder for each citizen/user/person, so the folders organization will be as following.
2.1 Face Detection
We will use the images of faces that have been divided from the videos or collected from the internet as input for the face detection algorithm. The algorithm takes the input data and detects faces in each image within the training folders, generating a .npz file which is a file format used in Python for storing and retrieving large numerical arrays and matrices.
For the face detection task in our system, we will be employing the MTCNN algorithm, which stands for Multi-Task Cascaded Convolutional Neural Network. This deep learning algorithm is commonly used for face detection and facial recognition tasks. MTCNN is a state-of-the-art algorithm that uses a cascading approach to detect faces and facial features such as the eyes, nose, and mouth. Below is an example of a Python code for face detection using MTCNN.
# Face Detection with MTCNN (Multi-Task Cascaded Convolutional Neural Network)
from os import listdir
from os.path import isdir
from PIL import Image
from matplotlib import pyplot
from numpy import savez_compressed
from numpy import asarray
from mtcnn.mtcnn import MTCNN
# extract a single face from a given photograph
def extract_face(filename, required_size=(160, 160)):
# load image from file
image = Image.open(filename)
# convert to RGB, if needed
image = image.convert('RGB')
# convert to array
pixels = asarray(image)
# create the detector, using default weights
detector = MTCNN()
# detect faces in the image
results = detector.detect_faces(pixels)
# check if face was detected
if results:
# extract the bounding box from the first face
x1, y1, width, height = results[0]['box']
# bug fix
x1, y1 = abs(x1), abs(y1)
x2, y2 = x1 + width, y1 + height
# extract the face
face = pixels[y1:y2, x1:x2]
# resize pixels to the model size
image = Image.fromarray(face)
image = image.resize(required_size)
face_array = asarray(image)
return face_array
else:
return None
# load images and extract faces for all images in a directory
def load_faces(directory):
faces = []
# enumerate files
for filename in listdir(directory):
# path
path = directory + filename
# get face
face = extract_face(path)
# check if face was detected
if face is not None:
# store
faces.append(face)
return faces
# load a dataset that contains one subdir for each class that in turn contains images
def load_dataset(directory):
X, y = [], []
# enumerate folders, on per class
for subdir in listdir(directory):
# path
path = directory + subdir + '/'
# skip any files that might be in the dir
if not isdir(path):
continue
# load all faces in the subdirectory
faces = load_faces(path)
# create labels
labels = [subdir for _ in range(len(faces))]
# summarize progress
print('>loaded %d examples for class: %s' % (len(faces), subdir))
# store
X.extend(faces)
y.extend(labels)
return asarray(X), asarray(y)
# load train dataset
trainX, trainy = load_dataset('/home/jawabreh/Desktop/face-recognition/training/')
print(trainX.shape, trainy.shape)
# save arrays to one file in compressed format
savez_compressed('/home/jawabreh/Desktop/face-recognition/face-detection.npz', trainX, trainy)MTCNN is composed of three main stages:
- Face Detection: In the first stage, the algorithm searches the image for regions that are likely to contain a face. This is done using a convolutional neural network (CNN) that is trained to identify faces in images.
- Facial Landmark Detection: In the second stage, the algorithm uses another CNN to locate the facial features, such as the eyes, nose, and mouth.
- Face Alignment: In the final stage, the algorithm aligns the detected face and facial features to a standardized pose, which is essential for accurate facial recognition.
If you explore other face recognition systems, you will notice that they typically include a step for face alignment before training. In contrast, MTCNN automatically performs face alignment for us, without the need for an additional step.
2.2 Face Features Extraction
Face feature extraction is a crucial step in face recognition systems. It involves transforming the raw image data of a face into a high-dimensional feature vector, which can be used to represent the unique facial characteristics of an individual. The feature extraction process aims to identify the most informative and discriminative facial features, such as the shape, texture, and color of facial components like eyes, nose, and mouth.
In our system we will use FaceNet algorithm which is a deep learning algorithm used for feature extraction in face recognition systems. It is a neural network model that maps a face image to a high-dimensional feature vector, which can be used for various face recognition tasks such as face verification, identification, and clustering. Below is an example of a Python code for face feature extraction using FaceNet, but before that we need to download the official version of FaceNet model, you can download it from this GitHub repo.
https://github.com/HumanexAI/HumanexAI_FaceRecognition
# Face Feature Extracion using FaceNet (calculating the face embedding)
from numpy import load
from numpy import expand_dims
from numpy import asarray
from numpy import savez_compressed
from keras.models import load_model
# get the face embedding for one face
def get_embedding(model, face_pixels):
# scale pixel values
face_pixels = face_pixels.astype('float32')
# standardize pixel values across channels (global)
mean, std = face_pixels.mean(), face_pixels.std()
face_pixels = (face_pixels - mean) / std
# transform face into one sample
samples = expand_dims(face_pixels, axis=0)
# make prediction to get embedding
yhat = model.predict(samples)
return yhat[0]
# load the face dataset
data = load('/home/jawabreh/Desktop/face-recognition/face-detection.npz')
trainX, trainy = data['arr_0'], data['arr_1']
print('Loaded: ', trainX.shape, trainy.shape, )
# load the facenet model
model = load_model('/home/jawabreh/Desktop/face-recognition/facenet_keras.h5')
print('Loaded Model')
# convert each face in the train set to an embedding
newTrainX = list()
for face_pixels in trainX:
embedding = get_embedding(model, face_pixels)
newTrainX.append(embedding)
newTrainX = asarray(newTrainX)
print(newTrainX.shape)
# save arrays to one file in compressed format
savez_compressed('/home/jawabreh/Desktop/face-recognition/embeddings/embeddings.npz', newTrainX, trainy)Unlike traditional feature extraction methods, which rely on hand-crafted features, FaceNet learns a feature representation directly from the raw face image data using a deep convolutional neural network architecture. This approach enables the algorithm to capture the unique facial features and variations that are crucial for accurate face recognition.
FaceNet has achieved state-of-the-art performance on various face recognition benchmarks, making it one of the most widely used algorithms for face recognition tasks in both academic research and industry applications.
By completing the face detection and feature extraction steps, we have concluded the second stage of our face recognition system. We can now utilize the output of the feature extraction process to carry out face identification.
3.0 Classification and identification
Face classification and identification is the final and most crucial step in a face recognition system. In this step, the system uses the extracted facial features to classify the face and determine the identity of the individual.
But before starting the face identification step, we must prepare the images that will be utilized for identification. In this regard, we require fresh data that has not been utilized to train the model. Additionally, we need to have several testing images at least tens for each class or individual. As such, the organization of the folders will be as follows:
This process involves comparing the facial features of the input image with the features in the system’s database, which contain information about known individuals. The system may use techniques such as feature matching, distance metrics, or machine learning algorithms to perform the classification and identification tasks.
In our system we will use SVM classifier for the identification, Support Vector Machines (SVMs) are one of the machine learning algorithms commonly used for face identification in the face classification and identification step of a face recognition system. SVMs are supervised learning models that analyze data and recognize patterns used for classification and regression analysis.
In face recognition, SVMs are used to classify the extracted facial features into different classes, i.e., different individuals. SVMs work by finding the hyperplane in a high-dimensional space that maximally separates the different classes of data. This hyperplane can be used to classify new, unseen data points into their corresponding classes.
SVMs have been shown to be effective in handling high-dimensional data, which is often the case in face recognition, making them a popular choice for this application. However, the performance of the SVM-based face identification system is highly dependent on the quality of the extracted features and the size and diversity of the training dataset. Below is an example of a Python code for classification and identification using SVM classifier.
import cv2
import numpy as np
from mtcnn import MTCNN
from tensorflow.keras.models import load_model
from sklearn.preprocessing import Normalizer, LabelEncoder
from sklearn.svm import SVC
import matplotlib.pyplot as plt
# Load face detection model
detector = MTCNN()
# Load FaceNet model
facenet_model = load_model('/home/jawabreh/Desktop/face-recognition/facenet_keras.h5')
# Load face embeddings
data = np.load('/home/jawabreh/Desktop/face-recognition/embeddings/embeddings.npz')
trainX, trainy = data['arr_0'], data['arr_1']
# Normalize input vectors
in_encoder = Normalizer(norm='l2')
trainX = in_encoder.transform(trainX)
# Label encode targets
out_encoder = LabelEncoder()
out_encoder.fit(trainy)
trainy = out_encoder.transform(trainy)
# Define the classes
class_names = out_encoder.classes_
class_names = np.append(class_names, 'unknown')
# Train SVM classifier
model = SVC(kernel='linear', probability=True)
model.fit(trainX, trainy)
# Define function to extract face embeddings
def extract_face_embeddings(image):
# Detect faces in the image
faces = detector.detect_faces(image)
if not faces:
return None
# Extract the first face only
x1, y1, width, height = faces[0]['box']
x2, y2 = x1 + width, y1 + height
face = image[y1:y2, x1:x2]
# Resize face to the size required by facenet model
face = cv2.resize(face, (160, 160))
# Preprocess the face for facenet model
face = face.astype('float32')
mean, std = face.mean(), face.std()
face = (face - mean) / std
face = np.expand_dims(face, axis=0)
# Generate embeddings using facenet model
embeddings = facenet_model.predict(face)
return embeddings[0]
# Define function to identify the identity of an input image
def identify_person(image):
# Extract face embeddings from input image
embeddings = extract_face_embeddings(image)
if embeddings is None:
return None, None
# Normalize embeddings
embeddings = in_encoder.transform([embeddings])
# Predict the identity and confidence using SVM classifier
prediction = model.predict(embeddings)
confidence = model.predict_proba(embeddings)[0][prediction] * 100
prediction = out_encoder.inverse_transform(prediction)
return prediction[0].item(), confidence
# Define function to identify the identity and confidence of an input image
def identify_person_with_unknown(image, threshold=0.9):
# Extract face embeddings from input image
embeddings = extract_face_embeddings(image)
if embeddings is None:
return None, None
# Normalize embeddings
embeddings = in_encoder.transform([embeddings])
# Predict the identity and confidence using SVM classifier
predictions = model.predict_proba(embeddings)[0]
max_idx = np.argmax(predictions)
if predictions[max_idx] >= threshold:
prediction = out_encoder.inverse_transform([max_idx])
confidence = predictions[max_idx] * 100
return prediction[0].item(), confidence
else:
return "unknown", None
# Example usage
image = cv2.imread('/home/jawabreh/Desktop/face-recognition/data/test/person_1/1.jpg')
person, confidence = identify_person_with_unknown(image)
if person is None:
print('No face detected in the input image!')
elif person == "unknown":
text = "Unknown person"
plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
plt.title(text)
plt.axis('off')
plt.show()
else:
# Display the predicted name and confidence probability
text = f'Predicted: {str(person)} ({confidence:.2f}%)'
plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
plt.title(text)
plt.axis('off')
plt.show()
If you examine the code, you will notice that we have replicated steps 2.1 and 2.2, namely face detection and face feature extraction. You may wonder why this is necessary, given that we have already executed these steps on the training data. The reason is that we need to repeat the face detection and feature extraction for the input image. We then take the resulting features from the input image and compare them to the features (embeddings) of the training classes. If the input image features are similar to any of the training class features, we can infer that it belongs to that particular class (person).
After finishing the data collection, training, and testing steps, the full pipeline of the system will be as follows:
To create both a training script and a testing (identification) script, we need to combine the face detection and face feature extraction algorithms. we already did that for the testing (identification), now we need to do that for the training script. The resulting training script will be as follows:
# Training a set of faces
from os import listdir
from os.path import isdir
from PIL import Image
from matplotlib import pyplot
from numpy import savez_compressed
from numpy import asarray
from numpy import load
from numpy import expand_dims
from numpy import asarray
from numpy import savez_compressed
from mtcnn.mtcnn import MTCNN
from keras.models import load_model
# Face detection using MTCNN
# extract a single face from a given photograph
def extract_face(filename, required_size=(160, 160)):
# load image from file
image = Image.open(filename)
# convert to RGB, if needed
image = image.convert('RGB')
# convert to array
pixels = asarray(image)
# create the detector, using default weights
detector = MTCNN()
# detect faces in the image
results = detector.detect_faces(pixels)
# check if face was detected
if results:
# extract the bounding box from the first face
x1, y1, width, height = results[0]['box']
# bug fix
x1, y1 = abs(x1), abs(y1)
x2, y2 = x1 + width, y1 + height
# extract the face
face = pixels[y1:y2, x1:x2]
# resize pixels to the model size
image = Image.fromarray(face)
image = image.resize(required_size)
face_array = asarray(image)
return face_array
else:
return None
# load images and extract faces for all images in a directory
def load_faces(directory):
faces = []
# enumerate files
for filename in listdir(directory):
# path
path = directory + filename
# get face
face = extract_face(path)
# check if face was detected
if face is not None:
# store
faces.append(face)
return faces
# load a dataset that contains one subdir for each class that in turn contains images
def load_dataset(directory):
X, y = [], []
# enumerate folders, on per class
for subdir in listdir(directory):
# path
path = directory + subdir + '/'
# skip any files that might be in the dir
if not isdir(path):
continue
# load all faces in the subdirectory
faces = load_faces(path)
# create labels
labels = [subdir for _ in range(len(faces))]
# summarize progress
print('>loaded %d examples for class: %s' % (len(faces), subdir))
# store
X.extend(faces)
y.extend(labels)
return asarray(X), asarray(y)
# load train dataset
trainX, trainy = load_dataset('/home/jawabreh/Desktop/face-recognition/data/train/')
print(trainX.shape, trainy.shape)
# save arrays to one file in compressed format
savez_compressed('/home/jawabreh/Desktop/face-recognition/face_detection/face_detection.npz', trainX, trainy)
# Face feature extraction using FaceNet
# get the face embedding for one face
def get_embedding(model, face_pixels):
# scale pixel values
face_pixels = face_pixels.astype('float32')
# standardize pixel values across channels (global)
mean, std = face_pixels.mean(), face_pixels.std()
face_pixels = (face_pixels - mean) / std
# transform face into one sample
samples = expand_dims(face_pixels, axis=0)
# make prediction to get embedding
yhat = model.predict(samples)
return yhat[0]
# load the face dataset
data = load('/home/jawabreh/Desktop/face-recognition/face_detection/face_detection.npz')
trainX, trainy = data['arr_0'], data['arr_1'],
print('Loaded: ', trainX.shape, trainy.shape, )
# load the facenet model
model = load_model('/home/jawabreh/Desktop/face-recognition/facenet_keras.h5')
print('Loaded Model')
# convert each face in the train set to an embedding
newTrainX = list()
for face_pixels in trainX:
embedding = get_embedding(model, face_pixels)
newTrainX.append(embedding)
newTrainX = asarray(newTrainX)
print(newTrainX.shape)
# save arrays to one file in compressed format
savez_compressed('/home/jawabreh/Desktop/face-recognition/face_embeddings/face_embeddings.npz', newTrainX, trainy)
print("\n\n\tFace Feature Extraction Done Successfuly\n\n")4.0 Optimization
If you noticed in the code, there is a value that if you change it the result will be totally different this value called threshold, in a face recognition systems, a threshold plays a critical role in determining whether an input image is a match for a particular individual in the system. Specifically, when using a Support Vector Machine (SVM) for face recognition, a threshold is used to classify a given input image into a particular class (person).
This threshold can be adjusted (optimized) to control the balance between false positives (incorrectly identifying a person) and false negatives (failing to identify a person). A high threshold is likely to result in fewer false positives, but it may also lead to more false negatives, while a low threshold can increase the likelihood of false positives, but reduce the number of false negatives. Therefore, it is crucial to carefully tune the threshold based on the specific requirements of the face recognition system to achieve the desired level of accuracy and efficiency.
In my dataset, the optimal threshold value is 0.9. However, the optimal threshold value for a different dataset may vary depending on various factors such as the size of the dataset, the quality of images, and the similarity between the faces in the dataset. Therefore, to achieve the highest accuracy for a specific dataset, the threshold value must be adjusted accordingly.
5.0 Evaluation
There are several metrics that can be used to evaluate a face recognition system, including precision, recall, and F1 score. Precision measures the proportion of correctly identified individuals out of all identified individuals, while recall measures the proportion of correctly identified individuals out of all individuals in the dataset.
5.1 Cross Validation
Cross-validation involves splitting the dataset into multiple subsets or folds, where each fold is used as a test set while the rest are used for training the model. The model is then evaluated on each fold, and the performance metrics are averaged across all folds.
Cross-validation helps to prevent overfitting and provides a more accurate estimate of the model’s performance. In the case of face recognition, cross-validation can be performed by splitting the dataset into groups of individuals, where each group contains a set of images for each individual. The model can then be trained on one group and tested on the remaining groups, and this process can be repeated for each group. By using cross-validation, we can obtain a more reliable estimate of the model’s accuracy and ensure that it is capable of generalizing well to unseen data.
Do we need to test the algorithm on each testing sample one by one? in the best practise testing a face recognition algorithm need hundreds of test samples doing that for tens of classes means thousands of attempts so ofcourse we will not dout manually we will make a small script to do that for us and give us a excel sheet with the result. Below is an example of a Python code for cross-validation test.
import cv2
import numpy as np
from mtcnn import MTCNN
from tensorflow.keras.models import load_model
from sklearn.preprocessing import Normalizer, LabelEncoder
from sklearn.svm import SVC
import pandas as pd
import os
# Load face detection model
detector = MTCNN()
# Load face recognition model
facenet_model = load_model('/home/jawabreh/Desktop/Face-Recognition/facenet_keras.h5')
# Load face embeddings
data = np.load('/home/jawabreh/Desktop/Face-Recognition/embeddings/Embeddings.npz')
trainX, trainy = data['arr_0'], data['arr_1']
# Normalize input vectors
in_encoder = Normalizer(norm='l2')
trainX = in_encoder.transform(trainX)
# Label encode targets
out_encoder = LabelEncoder()
out_encoder.fit(trainy)
trainy = out_encoder.transform(trainy)
# Define the classes
class_names = out_encoder.classes_
class_names = np.append(class_names, 'unknown')
# Train SVM classifier
model = SVC(kernel='linear', probability=True)
model.fit(trainX, trainy)
# Define function to extract face embeddings
def extract_face_embeddings(image):
# Detect faces in the image
faces = detector.detect_faces(image)
if not faces:
return None
# Extract the first face only
x1, y1, width, height = faces[0]['box']
x2, y2 = x1 + width, y1 + height
face = image[y1:y2, x1:x2]
# Resize face to the size required by facenet model
face = cv2.resize(face, (160, 160))
# Preprocess the face for facenet model
face = face.astype('float32')
mean, std = face.mean(), face.std()
face = (face - mean) / std
face = np.expand_dims(face, axis=0)
# Generate embeddings using facenet model
embeddings = facenet_model.predict(face)
return embeddings[0]
# Define function to identify the identity of an input image
def identify_person(image):
# Extract face embeddings from input image
embeddings = extract_face_embeddings(image)
if embeddings is None:
return None, None
# Normalize embeddings
embeddings = in_encoder.transform([embeddings])
# Predict the identity and confidence using SVM classifier
prediction = model.predict(embeddings)
confidence = model.predict_proba(embeddings)[0][prediction] * 100
prediction = out_encoder.inverse_transform(prediction)
return prediction[0].item(), confidence
# Define function to identify the identity and confidence of an input image
def identify_person_with_unknown(image, threshold=0.9):
# Extract face embeddings from input image
embeddings = extract_face_embeddings(image)
if embeddings is None:
return None, None
# Normalize embeddings
embeddings = in_encoder.transform([embeddings])
# Predict the identity and confidence using SVM classifier
predictions = model.predict_proba(embeddings)[0]
max_idx = np.argmax(predictions)
if predictions[max_idx] >= threshold:
prediction = out_encoder.inverse_transform([max_idx])
confidence = predictions[max_idx]
return prediction[0].item(), confidence
else:
return None, None
# Define path to directory containing images
image_dir = '/home/jawabreh/Desktop/Face-Recognition/data/test/Jawabreh/'
# Create an empty DataFrame to store the results
results_df = pd.DataFrame(columns=['filename', 'predicted_identity', 'confidence'])
# Iterate through each image in the directory
for filename in os.listdir(image_dir):
# Read the image
image_path = os.path.join(image_dir, filename)
image = cv2.imread(image_path)
# Call the identify_person_with_unknown function to get the predicted identity and confidence
predicted_identity, confidence = identify_person_with_unknown(image)
# Add the results to the DataFrame
results_df = results_df.append({'filename': filename, 'predicted_identity': predicted_identity, 'confidence': confidence}, ignore_index=True)
# Save the results to an Excel sheet
results_df.to_excel('/home/jawabreh/Desktop/Face-Recognition/crossValidation-results.xlsx', index=False)
5.2 Confusion Matrix
A confusion matrix is a useful tool for evaluating the performance of a face recognition system. The matrix provides a summary of the system’s predictions and their actual outcomes, highlighting the number of true positive, true negative, false positive, and false negative predictions. Below is an example of a Python code forconfusion matrix test.
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
# Define the confusion matrix
# the values here is the number of correct identification
confusion_matrix = [[49, 0, 0, 1], [0, 34, 0, 0], [0, 0, 44, 0], [0, 0, 0, 50]]
# Define the class labels
labels = ["Person_1", "Person_2", "Person_3", "Person_4"]
# Create a DataFrame from the confusion matrix and labels
df_cm = pd.DataFrame(confusion_matrix, index=labels, columns=labels)
# Plot the heatmap using Seaborn
sns.heatmap(df_cm, cmap="Blues", annot=True, fmt="d")
# Add axis labels and a title
plt.xlabel("Predicted Label")
plt.ylabel("True Label")
plt.title("Confusion Matrix")
# Show the plot
plt.show()5.3 F1, Recall, and Precesion Scores
F1 score, recall, and precision scores are commonly used metrics to evaluate the performance of a face recognition system. The precision score measures the accuracy of the system in correctly identifying positive instances (i.e., correctly identifying individuals) out of all instances that the system has identified as positive. The recall score measures the ability of the system to correctly identify positive instances out of all actual positive instances in the dataset.
The F1 score is the harmonic mean of precision and recall, providing a comprehensive measure of the system’s performance. In the context of face recognition, precision, recall, and F1 score can be used to assess the system’s ability to accurately identify individuals in the dataset. A high precision score indicates that the system has a low rate of false positives, while a high recall score indicates that the system has a low rate of false negatives. A high F1 score indicates that the system has high accuracy in both identifying positive instances and avoiding false positives and negatives, making it a useful metric for evaluating the overall performance of a face recognition system.
- Precision = TP / (TP + FP)
- Recall = TP / (TP + FN)
- F1 Score = 2 * (Precision * Recall) / (Precision + Recall)
5.4 ROC Curve
A receiver operating characteristic (ROC) curve is a graphical representation of the performance of a face recognition system. The curve plots the true positive rate (TPR) against the false positive rate (FPR) at different threshold levels. In the context of face recognition, the ROC curve can be used to evaluate the system’s ability to distinguish between positive and negative instances (i.e., identifying individuals correctly and rejecting impostors). Below is an example of a Python code for ROC Curve test.
import matplotlib.pyplot as plt
# Define the data
thresholds = [0.5, 0.6, 0.7, 0.8, 0.85, 0.9, 0.95]
tpr = [150/178, 154/178, 164/178, 171/178, 174/178, 176/178, 173/178]
fpr = [28/178, 24/178, 14/178, 7/178, 4/178, 1/178, 5/178]
# Create a figure with two axes
fig, (ax1, ax2) = plt.subplots(2, 1, sharex=True)
# Plot the TPR on the top axis
ax1.plot(thresholds, tpr, marker='o', markersize=5, color="blue")
ax1.set_ylabel('True Positive Rate')
# Plot the FPR on the bottom axis
ax2.plot(thresholds, fpr, marker='o', markersize=5, color="red")
ax2.set_xlabel('Threshold')
ax2.set_ylabel('False Positive Rate')
# Show the plot
plt.show()Conclusion
In conclusion, face recognition technology has come a long way in recent years, and it has become an essential tool for various industries, including law enforcement, security, and marketing. However, the technology is not without its challenges and limitations, including issues with accuracy, bias, and privacy concerns.
It is essential to address these challenges to ensure that face recognition technology is used responsibly and ethically. Nonetheless, the potential benefits of the technology are significant, and with continued research and development, we can expect to see even more advances in the field. As the technology continues to evolve, it is crucial that we remain vigilant and thoughtful in our approach, balancing the benefits of face recognition with its potential risks and limitations.The project github repo: https://github.com/Jawabreh0/HumanexAI_FaceRecognition
My GitHub for more projects: https://github.com/Jawabreh0
