Real-Time Detect Profile on Chrome NodeJS! #3 FINAL

Photo by Waldemar Brandt on Unsplash

It’s the final part of a series on how to make the face API to use with a profile identifier, that can recognize who is using the computer and open the specified Chrome profile.

To start please visit here https://blog.hiltonws.com/face-detector-with-nodejs-profile-identifier-1-70889ac0bf3c.

Second part here https://blog.hiltonws.com/face-recognition-with-nodejs-profile-identifier-2-e0adacdcc0c6

What we will do here
1- Detect Face
2- Detect Face in Real-time (run)
3- Process Frame
4- Check If Chrome was Opened
5- Glue Together.

Detect Face

As we already learn, we can detect faces using images, lets create a new file called profileRecognition.js, let’s start with the following template

There is the structure we will use to process the images in real-time.

The first part is the required libs

Don’t forget to install each lib with

npm install libName

Where libName is the lib of each require.

Questions? See here https://blog.hiltonws.com/first-api-with-node-js-and-express-da7ade41ca39

With this setup let’s start creating our

async function detectFace(fullFaceDescriptions, label, videoCapture)

Create these variables inside the function

//If matches stops loooping to next file
let match;
// Compare folder
let compare = ‘./images/’ + label + ‘/’;

The compare folder will be a folder where the images were captured on previous posts. The label should be an existed folder, which matches with the label that we will find.

So, we will use fs functions to read from the folder

With it, we can find each image file from the sample folder to use in the next steps

Here we will decode the image to use with tensorFlow.js, and with the tensor frame, we can find face descriptor, which marks the mainly characteristics from the detected face.

Adding more code.

The above code, compare descriptor from a detected image, and descriptors received by a parameter called fulfaceDescriptions, which contains one or more faces descriptors.

Now, we search in results the bestMatch, with the following

As you can see the code already add the following

It starts an independent process releasing the video source and exit from the script, a process opens chrome with the Default profile that’s matched to Hilton’s label, also add a restore-last-session parameter, to open the last session.

The final function should be something like that

We have some dependencies to run the function so let’s code a little more.

More questions? Here https://blog.hiltonws.com/face-recognition-with-nodejs-profile-identifier-2-e0adacdcc0c6

Detect Face in Real-time (run)

Here we will create the function

async function run()

Inside the function, we will capture each frame from our webcam. But first, import the faceapi.js trained models.

//Loading models
await faceDetectionLandmark68.loadFromDisk(MODELS);
await faceRecognition.loadFromDisk(MODELS);
await ssdMobilenetv1.loadFromDisk(MODELS);

With all models imported, we will create our video source “manipulator”.

//Attach a video device
let videoCapture = new cv.VideoCapture(0);

So, now we can capture images from our videoCapture device.

We called by first time the capture on line 19 after it the function will call processFrame that's calls capture when all process is finished without results.

On lines 7 and 9, we are update globalframe with a buffered data which represents the image, that is processed by processFrame

All function will look like

Process Frame

Process frame is a function consists of the following signature

async function processFrame(capture, videoCapture)

We are seeing the parameters, capture, and videoCapture. The capture parameter will be a function that reads each frame from our webcam, this will be called again at end of processFrame.The videoCapture parameter will be used to release the webcam when we found a labeled face. As we saw on detectFace function.

Here is all the function

It’s a short function, which basically creates a fullFaceDescriptions (line 9), to be used on detectFace function, so we receive each frame from globalFrame and detectAllFaces from there.

If chrome was not opened from detectFace function, we will capture a new frame.

Check If Chrome was Opened

This function prevents open multiple instances of Chrome and prevents create a new window if the chrome is already open.

This calls a child process wich checks if has any instance with name chrome, if has any instance of Chrome the process exit, that means the NodeJs exit the script. It’s can be adapted to be run by different operating systems, in the code is used Linux only.

Glue Together

Here is the easiest part, put this on end of file

closeIfChromeIsRunning();
run()

All code should be like this

Please see this part of the article in
https://github.com/HiltonWS/profileIdentifier/

Thank you for stay here and see all the series! I love to write about my experiences, so probably, I see you in the next post.