face-verify.js: Monitoring who is physically looking at a website for additional security
tl;dr: This is a demo project showing how Machine Box tech can be integrated into JavaScript applications.
Facebox can take an image and tell you how many faces it sees, as well as who those faces belong to provided you have shown it a single example previously.
We can use this capability to build additional security into our web apps so we can see how many people are watching the screen and who they are. Using the webcam with some JavaScript and Facebox, we can periodically check to ensure only authorised people can see the information that users consider sensitive.
A working demo of this project is available: Check out the Face Verify project on GitHub
Use case
Banks don’t want private account details (like the user’s current balance and credit limits etc) being seen by anybody other than the account holder.
The Face Verify technique can be used to automatically hide sensitive information if:
- Facebox doesn’t recognise the person looking at the screen, or
- if multiple people are in front of the screen
Ethical implications
This kind of system should only be implemented with user’s consent and if they choose to enable this kind of capability for themselves. In Face Verify, the user has to give the website permission to access the webcam before it will even work, and without a reference image in the first place, Facebox cannot identify them anyway.
Like any technology, we need to consider the ethics of its application carefully so we don’t build tools that are open to abuse, or worst case, terminators that can travel through time to kill people.
Example
In the following screen, I have moved out of view of the Face Verify tool, and the sensitive information is blacked out:
When I move into focus, the information is revealed:
The project
The Face Verify project has a JavaScript SDK that provides this capability, and it’s very simple to use:
After spinning up an instance of Facebox (in production you would deploy instances and probably configure them to respond elasticity to demand) you just need to create an instance of the machinebox.FaceVerify class, and call start().
The onSecure and onInsecure callbacks are triggered when the state changes between the page being secure (i.e. only one authorized person is looking at the screen) and being insecure (Facebox doesn’t recognize the person, or if there are multiple people).
The snapshotInterval is how long to wait between checks in milliseconds, so 1000 is one second.
Open source
The solution is open-source (with an Apache license) and you can see the full implementation by checking out the face-verify.js file.
Is this really secure?
The “additional security” headline was chosen deliberately. Obviously you wouldn’t want to rely on this technique as the only security layer but for some circumstances it’s a great way to ensure only the right people are in front of the computer.
There are a number of ways this kind of capability could be circumnavigated, and a few in which it could be abused, which is why we rely on browsers to ensure this kind of thing is only done with the user’s consent.
What is Machine Box?
Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.
The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.
Have a play and let us know what you think.
