How I built Face Unlock for Ubuntu Linux
Introduction
This post is about how I built face unlock or as Apple might say “face ID” for my Ubuntu machine. Let’s get started!
Objective
It aims to build a face authentication system that can work at the Login time and while running the “sudo” or “su” commands in Ubuntu.
Let’s breakdown the problem first:
- How to build face recognizer
- How to add this feature to Ubuntu
Implementation
1. How to build face recognizer
I used OpenCV and the fastai library to build my face recognizer. So first of all just to build the model, I used my face pictures and my friends’ face pictures and then I transfer learned the model with the Vgg16 pre-trained model. And actually, it didn’t give me satisfactory accuracy. Then I realized that there is much noise in the images and the pictures are very wide instead of just being face images. So I used OpenCV’s “haarcascade_frontalface_default.xml” to extract the faces from the pictures and then again trained the model and tried some combinations of parameters and it suddenly got very nice accuracy on the validation dataset(>93%).
I had built my model and decided my parameter tuning for the final model but the real-time face recognization was not implemented yet. Then I wrote code to click a picture, to extract the face portion and then return it to the caller function and another function to call the above function and feed the returned picture to the model.
I found out that this time it was mostly giving me either wrong output or correct output with lower confidence. I realized that my laptop’s camera clicks far worse pictures than the once I used for training so I decided to click all the pictures on my pc with the same code I used for returning a picture to the model but with some different set of parameters. Like if it is for testing the function call will be getFaces() but for training purposes, it will be getFaces(training = True). And then my model started giving good accuracy again :)
And phase 1, the Deep learning part was almost done but the harder part was remaining though which was adding this to Ubuntu.
2. How to add this feature to Ubuntu
Subtasks:
- The research phase
- Deployment phase
- Few final touches
The research phase
I know it seems very difficult to make a change in any OS but the Linux guys have made Linux so dynamic that to make changes or to add something in any Linux distribution is not so tough. All you need is the ability to do some research and patience.
When I got an idea about adding a Face unlocking system in my Ubuntu machine I was thinking that this is gonna be super difficult and I will have to make changes in the Linux kernel’s code. But after some days of research about the authentication system of Linux, I came to know about PAM(Pluggable Authentication Modules) from this page. And then I realized how dynamic the Linux is!
PAM is a C++ Interface and whoever implements that interface becomes an Authentication method. So the file implementing the PAM interface can become an authentication method by just adding one extra line in the file “/etc/pam.d/common-auth” which are set of rules for authentication in Ubuntu Linux.
But here comes the trickier part. the PAM interface is written in C++ and I had decided to implement the face recognization in python so I couldn’t figure out a way to handle this situation. I don’t know how but Michelly I came to know about pam_python while searching for something. And pam_python is a bridge between PAM interface C++ and python language.
Deployment phase
So with all this research, I installed pam_python and put the pam_python.so file into the “/lib/x86_64-linux-gnu/security/” directory. See in the below image.
Then I implemented the PAM interface taking help of pam_python, and implemented 4 functions that had to be implemented which are,
(1) pam_sm_authenticate(pamh, flags, args)(for “sudo” commands)
(2) pam_sm_open_session(pamh, flags, args)(for opening sessions like login and “su” command)
(3) pam_sm_close_session(pamh, flags, args) (for closing the sessions)
(4) pam_sm_setcred(pamh, flags, args)(to set the credentials of the current user associated with the authentication handle).
Then for adding it to the list of authentication methods I had to make a rule file like the once in /etc/pam.d/ directory and put that into /usr/share/pam-configs/ directory.
And Boom!! it started working. There were many difficult situations in making this though. I almost corrupted my whole system twice! But the recovery mode saved me :)
Few final touches
I wanted to make it simple to install for others so I wrote shell scripts so that anyone can install it in 5–6 commands which are mentioned in the README there. And I wrote shell scripts to add a command named “facerec” to the system which can be used to enable the face unlock and to add a new-extra face model to the system like “facerec enable” and “facerec new”.
The face unlock runs perfectly alright while logging in and while running “sudo” or “su” commands. The process is a bit slow though. Below is the link to my GitHub repo of this project.
Anyone can install this by just running 5–6 commands in their system. Which are written there in the README, feel free to go there and install it. Contributions and issues or any new ideas are most welcome. Because I have further plans for it like,
1. Adding parallel processing for more speed.
2. Changing the algorithm, and making an unsupervised algorithm using autoencoder and decoder technique and using this dataset. Which I think can become a much more efficient algorithm in terms of classification.
3. As it’s only for the Ubuntu distribution now adding support for other distros is also on the list.
Conclusion
So this way you can also add your favorite features in Linux distribution of your choice. All you just need is research and patience.
PS: I am looking for an internship in ML/DL. Checkout my (GitHub),(Kaggle) and (LinkedIn) profiles.
Keep reading, Keep hacking — Thank you :)
