Build your own face recognition web app using face-api.js

Face recognition

Face recognition is a technology that identifies or verifies individuals by analyzing and comparing their facial features with a database of known faces. It goes beyond face detection, which simply detects the presence of faces in images or videos.

Face recognition involves several steps:

1. Face Detection: Face detection algorithms initially detect faces in an image or video frame. This step locates and extracts the facial regions.
2. Feature Extraction: The facial features of the face’s position, shape, and texture are extracted. These features are often represented as mathematical vectors or templates, capturing unique characteristics that distinguish one face from another.
3. Database Comparison: The extracted facial features are compared with a pre-existing database of known faces. The database may contain a set of templates or feature vectors associated with specific individuals.
4. Matching and Identification: The system compares the extracted features with the database to find the closest match or matches. If a match is found, the identity of the person associated with the matched template is determined. In cases where the system is designed for verification purposes, the match is used to confirm or reject the claimed identity of an individual.

In this blog post, we will deep dive into how to build your own face recognition web app using the face-api.js library. With face-api.js, you can easily detect and recognize faces in images or real-time video streams right in the browser. Let’s dive in and create our own face recognition web app!

Prerequisites

To follow along with this tutorial, you should have a basic understanding of HTML, CSS, and JavaScript. In case you are new to JavaScript don’t worry I have mentioned the GitHub link to the whole code implementation which uses Next JS and also attached a link to a live demo where you can play around.

Let’s dive into the coding part.

Load models using face-api.js

To load models first you need to download the required model files.
Find all the required models in my GitHub repository. After downloading we need to load these models using face-api.js.

import {
    loadFaceLandmarkModel,
    loadFaceRecognitionModel,
    loadSsdMobilenetv1Model,
} from 'face-api.js';

const MODEL_URL = '/models';

const loadModels = async () => {
    await loadSsdMobilenetv1Model(MODEL_URL);
    await loadFaceLandmarkModel(MODEL_URL);
    await loadFaceRecognitionModel(MODEL_URL);
};

useEffect(() => {
    loadModels();
}, []);

We have loaded 3 models

1. Mobilenet SSD: Used for detecting faces and getting bounding box data to highlight the faces in an image.
2. Face Landmark: Used for extracting data like face position, shape, and texture.
3. Face Recognition: Used for comparing facial features with existing data from the database.

Extract face data from the image to create a database

We will need some image database with facial features data to compare and recognize the person in the image. For that, we will use face-API functions like those given in the below example.

import {
    FaceMatcher,
    LabeledFaceDescriptors,
    detectSingleFace,
} from 'face-api.js';

const FACE_MATCHER_THRESHOLD = 0.6;

const processImagesForRecognition = async () => {
    let labeledFaceDescriptors = [];
    labeledFaceDescriptors = await Promise.all(
        imageElements?.map(async (imageEle) => {
            if (imageEle) {
                const label = imageEle?.alt.split(' ')[0];
                const faceDescription = await detectSingleFace(imageEle)
                    .withFaceLandmarks()
                    .withFaceDescriptor();
                if (!faceDescription) {
                    throw new Error(`no faces detected for ${label}`);
                }

                const faceDescriptors = [faceDescription.descriptor];
                return new LabeledFaceDescriptors(label, faceDescriptors);
            }
        })
    );

    const faceMatcher = new FaceMatcher(
        labeledFaceDescriptors,
        FACE_MATCHER_THRESHOLD
    );
};

We are iterating through the uploaded images for creating the database. We have used the detectSingleFace function for detecting a face from an image, then extracting facial features using withFaceLandmarks and converting them into readable descriptions using withFaceDescriptor method. At last, we are creating an instance of the faceMatcher using FaceMatcher constructor by providing all data extracted from our images.

Recognize face from Image

We have extracted data from images to create a database for recognition. Now we have to implement recognition logic for our query image or image from which the faces need to be recognized.

import {
    detectAllFaces,
    draw,
    matchDimensions,
    resizeResults,
} from 'face-api.js';

const loadRecognizedFaces = async () => {
    const resultsQuery = await detectAllFaces(queryImageElement.current)
        .withFaceLandmarks()
        .withFaceDescriptors();

    await matchDimensions(
        queryCanvasElement.current,
        queryImageElement.current
    );

    const results = await resizeResults(resultsQuery, {
        width: (queryImageElement.current as HTMLImageElement).width,
        height: (queryImageElement.current as HTMLImageElement).height,
    });

    const queryDrawBoxes = results.map((res) => {
        const bestMatch = faceMatcher.findBestMatch(res.descriptor);
        return new draw.DrawBox(res.detection.box, {
            label: bestMatch.toString(),
        });
    });

    queryDrawBoxes.forEach((drawBox) =>
        drawBox.draw(queryCanvasElement.current as unknown as HTMLCanvasElement)
    );
};

In the above code, we have used the detextAllFaces method by chaining it withFaceLandmarks and withFaceDescriptors to detect and extract facial feature data from the image.

We will use canvas to highlight detected faces by drawing a rectangle around them. First, we need to fetch the dimensions of our image element and canvas element to resize the canvas as per the image element. By doing this we make sure that we are drawing the bounding box in the correct position.

Now we find the best match from our database created earlier using the findBestMatch method from the faceMatcher instance.

const bestMatch = faceMatcher.findBestMatch(res.descriptor);

Finally, we draw the bounding box over the canvas using data from bestMatch object. The final result should highlight the faces with name labels in the image.

Conclusion

Congratulations! You have successfully built a face recognition web app using face-api.js. The above code is written in NextJS, but you can take references and build your web app in any JavaScript framework. In the references below you can find links for code implementation and a link to the demo.

Happy coding!

Reference

GitHub code for face recognition: https://github.com/NSV1991/nextjs-portfolio/tree/main/src/pages/work/face-recognition

Live demo for face recognition: https://neerajvageele.com/work/face-recognition

Find more from my work
https://neerajvageele.com/work

GitHub code for face-api.js: https://github.com/justadudewhohacks/face-api.js/tree/master