Real-Time Detect Profile on Chrome NodeJS! #3 FINAL
It’s the final part of a series on how to make the face API to use with a profile identifier, that can recognize who is using the computer and open the specified Chrome profile.
To start please visit here https://blog.hiltonws.com/face-detector-with-nodejs-profile-identifier-1-70889ac0bf3c.
Second part here https://blog.hiltonws.com/face-recognition-with-nodejs-profile-identifier-2-e0adacdcc0c6
What we will do here
1- Detect Face
2- Detect Face in Real-time (run)
3- Process Frame
4- Check If Chrome was Opened
5- Glue Together.
Detect Face
As we already learn, we can detect faces using images, lets create a new file called profileRecognition.js, let’s start with the following template
There is the structure we will use to process the images in real-time.
The first part is the required libs
Don’t forget to install each lib with
npm install libName
Where libName is the lib of each require.
Questions? See here https://blog.hiltonws.com/first-api-with-node-js-and-express-da7ade41ca39
With this setup let’s start creating our
async function detectFace(fullFaceDescriptions, label, videoCapture)
Create these variables inside the function
//If matches stops loooping to next file
let match;// Compare folder
let compare = ‘./images/’ + label + ‘/’;
The compare folder will be a folder where the images were captured on previous posts. The label should be an existed folder, which matches with the label that we will find.
So, we will use fs functions to read from the folder
With it, we can find each image file from the sample folder to use in the next steps
Here we will decode the image to use with tensorFlow.js, and with the tensor frame, we can find face descriptor, which marks the mainly characteristics from the detected face.
Adding more code.
The above code, compare descriptor from a detected image, and descriptors received by a parameter called fulfaceDescriptions, which contains one or more faces descriptors.
Now, we search in results the bestMatch, with the following
As you can see the code already add the following
It starts an independent process releasing the video source and exit from the script, a process opens chrome with the Default profile that’s matched to Hilton’s label, also add a restore-last-session parameter, to open the last session.
The final function should be something like that
We have some dependencies to run the function so let’s code a little more.
More questions? Here https://blog.hiltonws.com/face-recognition-with-nodejs-profile-identifier-2-e0adacdcc0c6
Detect Face in Real-time (run)
Here we will create the function
async function run()
Inside the function, we will capture each frame from our webcam. But first, import the faceapi.js trained models.
//Loading models
await faceDetectionLandmark68.loadFromDisk(MODELS);
await faceRecognition.loadFromDisk(MODELS);
await ssdMobilenetv1.loadFromDisk(MODELS);
With all models imported, we will create our video source “manipulator”.
//Attach a video device
let videoCapture = new cv.VideoCapture(0);
So, now we can capture images from our videoCapture device.
We called by first time the capture on line 19 after it the function will call processFrame that's calls capture when all process is finished without results.
On lines 7 and 9, we are update globalframe with a buffered data which represents the image, that is processed by processFrame
All function will look like
Process Frame
Process frame is a function consists of the following signature
async function processFrame(capture, videoCapture)
We are seeing the parameters, capture, and videoCapture. The capture parameter will be a function that reads each frame from our webcam, this will be called again at end of processFrame.The videoCapture parameter will be used to release the webcam when we found a labeled face. As we saw on detectFace function.
Here is all the function
It’s a short function, which basically creates a fullFaceDescriptions (line 9), to be used on detectFace function, so we receive each frame from globalFrame and detectAllFaces from there.
If chrome was not opened from detectFace function, we will capture a new frame.
Check If Chrome was Opened
This function prevents open multiple instances of Chrome and prevents create a new window if the chrome is already open.
This calls a child process wich checks if has any instance with name chrome, if has any instance of Chrome the process exit, that means the NodeJs exit the script. It’s can be adapted to be run by different operating systems, in the code is used Linux only.
Glue Together
Here is the easiest part, put this on end of file
closeIfChromeIsRunning();
run()
All code should be like this
Please see this part of the article in
https://github.com/HiltonWS/profileIdentifier/
Thank you for stay here and see all the series! I love to write about my experiences, so probably, I see you in the next post.