Face Recognition — The browser’s knack!
What if I told you that you could have eyes on your back too when you are dealing with some sensitive information on your laptop? What if you could blur out your data if anyone is peeking into your laptop? Imagine if you are able to see activities happening behind your back while looking at your laptop screen. Sounds impossible, doesn’t it? Well, it’s not. I am here to explain how you can do all of this using just your browser!
Do you feel this might be another article which will give you a solution that has high implementation costs, lots of data storage, etc? Put your worries to rest. I will show you a solution that is entirely on the client-side and in the browser!
OK, enough of the glitz! Let’s see what we are talking about..
With face-api.js, we have come up with a face recognition enabled smart banking dashboard.
What is face recognition enabled smart dashboard, you ask?
Face Recognition enabled — smart dashboard is a use-case related to information security of a banking application dashboard which has been developed by the smart UI initiative team at Globant India demonstrated in UINxt 2020 event.
What does this smart dashboard do exactly which makes it smart enough?
This dashboard is smart in both ways, in its technical implementation as well as in its use case point of view.
From a technical perspective its smart because it does face-recognition completely in browsers at client side
From a user perspective, it provides better information security to the user’s confidential data.
How does a smart dashboard work?
If a person (authorized user) has logged in into a bank account watching his/her dashboard data and suddenly any person other than logged in user arrives unknowingly from behind to watch data, at that time only, this smart dashboard recognizes an intruder or other person into the frame and get blurs out automatically by itself and protect user confidential data.
OK, time to see some code!
This smart dashboard is built with face-api.js and has used various APIs available in it and models to pre-train with user images.We used various APIs from face-api for
1. Detecting faces -
Detect all faces in an image and returns Array <FaceDetection>
Detect single face with the highest confidence score in an image and returns FaceDetection | undefined
2. Detecting 68 Face Landmark Points -
After face detection, we predict the facial landmarks for each detected face.
Detect all faces in an image + computes 68 point face landmarks for each detected face and return an array.
Detect the face with the highest confidence score in an image + computes 68 Point Face Landmarks for that face. Returns WithFaceLandmarks<WithFaceDetection<{}>> | undefined:
3. Computing Face Descriptors -
After face detection and facial landmark prediction the face descriptors for each face can be computed as follows:
Detect all faces in an image + compute 68 Point Face Landmarks for each detected face. Returns Array<WithFaceDescriptor<WithFaceLandmarks<WithFaceDetection<{}>>>>:
Detect the face with the highest confidence score in an image + compute 68 Point Face Landmarks and face descriptor for that face. Returns
WithFaceDescriptor<WithFaceLandmarks<WithFaceDetection<{}>>> | undefined:
4. Face Recognition by Matching Descriptors -
To perform face recognition, we use faceapi.FaceMatcher to compare reference face descriptors (descriptors of faces detected through webcam video input) to query face descriptors(descriptors of faces detected in images provided to pre-trained model).
The face matcher uses a Euclidean Distance as a similarity metric between two face descriptors.
Let’s start putting everything together now!
Step 1: Include face-api.js
It is very simple to include face-api.js you just need to include it using the script tag. Simply include the latest script from dist/face-api.js
Step 2 : Load Models
Load models provided by face-api.js via faceapi.nets. Depending on the requirements of your application you can specifically load the models you need, but to run a full end to end this we will need to load the face detection, face landmark and face recognition model.
The model files can simply be provided as static assets in your web app or you can host them somewhere else and they can be loaded by specifying the route or URL to the files. Let’s say you are providing them in a models directory along with your assets under public/models
Step 3: Creating a Canvas Element from an Image or Video Element.
Step 4: Creating Labelled Face Descriptors
Step 5: Comparing Face Descriptors & Recognizing Face
Now that we have the recognition done, providing a ‘sensitiveInformattion’ class to the HTML elements that contain sensitive information. We then switched a ‘textshadow’ class on that element to blur sensitive data when an intruder is detected.
Let’s put this together to form a basic setup of face recognition in your app
Alright! We are all set with the code. Now for the exciting demo!
Here you can see, a person is coming from behind and peeking into the laptop screen of the user. As soon as the dashboard recognizes another person(intruder) into frame it automatically blurs out user’s sensitive data like account balance, loan information, etc.
That’s all smart dashboard is about, I’m off now and I hope you find this interesting…I would love to hear from you …please do share your feedback into the comments below.
Have a fantastic week & Happy Reading !!!
