Smart Attendance Management System using Intel DevCloud
Facial and voice recognition-based Smart attendance system
Facial recognition could soon jump from your smartphone to your workplace with employers using it to mark attendance and gauge the mood of the workforce. Every day, corporate offices and institutes are working to increase the productive working hours in a day. When the current system of clocking in daily using a fingerprint scanner is a time-consuming and inefficient use of time.
I have planned to design a Voice Interactive Enabled Face Detection Based Smart Attendance management and behavior analysis to ensure a better work culture and environment, efficiency in a secure manner using Intel DevCloud. Currently, we have fingerprint and smart card-based entries in nearly all offices and a few schools and colleges. These systems then automate the calculation of salary or attendance percentage. But fingerprint scanning and smart card barcode entries tend to take up time and prove to also be imperfect. In contrast, Face Recognition method provides a unique feature for every individual which is stored in a central database and can be retrieved during recognition and validation.
The system includes an embedded application deployed in an SCB( Single Board Computer) which can interact with the users in real time. It will take down in and out time of every employee and monitor their working behavior (future scope) and notify the corresponding employee and the authority at times. We are aiming to analyze people’s behavior, mood, and emotions by monitoring and studying their actions in real time which in turn will help the organization know about the physical and mental status of the employees. This process of direct integration of the physical world into computer vision-based systems will indeed result in efficiency improvements, economic benefits and reduced human exertions.
As of now, I have developed a basic voice interactive attendance monitoring using Jupyter Notebook on Intel dev cloud. The in and out time (including mid in and out) will be monitored in a Google spreadsheet and the system will calculate how many hours an employee has spent in office premises. The system won’t allow employees to step into the office after a certain time and won’t consider the attendance if the total hours spent is less than four hours. Every day a mail will be sent to the admin containing the attendance details of the employees.
In future, I would like to implement behavior and mood analysis of the employees and the staff on the office premises which in turn will help the concerned staff provide with solutions to get over the listless mood or erratic behavior.
Design Methodology:
Despite significant recent advances in the field of face recognition, implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. Here, we present a system called FaceNet that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings as feature vectors. Our method uses a deep convolutional network trained to directly optimize the embedding itself, rather than an intermediate bottleneck layer as in previous deep learning approaches. To train, we use triplets of roughly aligned matching / non-matching face patches generated using a novel online triplet mining method. The benefit of our approach is much greater representational efficiency: we achieve state-of-the-art face recognition performance using only 128-bytes per face.
In Corporate offices that have face recognition based attendance, the database of employee facial analytics data is first created and registered (using face detection algorithm). Once this data is compiled, the smart CCTV cameras powered by Face Vector APIs (or SDKs) are positioned appropriately at the office entrances and exits and scans incoming or outgoing crowds. Face detection will detect all faces in the frame and Face Verification/ Face Vector will compare the detected facial attributes to the data stored in the database and ID the incoming crowd. The face recognition system IDs the employees and clocks them in for that day. The software first captures an image of all the authorized persons and stores the information into a database. The system then stores the image by mapping it into a face coordinate structure. Next time whenever the registered person enters the premises the system recognizes the person and marks his attendance along with the time. If the person arrives later than his reporting time, the system speaks a warning “you are xx minutes late! Do not repeat this.” The face detection software has a few more options for managing various critical situations such as- providing a late option for those employees who enter late, taking necessary actions like 3 successive late would cost the employee one casual leave. If someone exits in the midway of office for lunch or anything, then the system would alert him through SMS/mail to come back to the office within a certain time, otherwise, the time at which he went out of office will be considered as his out time. The user can interact with the system through his voice for any further assistance such as-showing user’s latest attendance report or any discrepancy. The primary benefit is that you cannot mark attendance by proxy. Factories and workplaces that employ blue-collar workers show the greatest demand for facial recognition and fingerprint-based biometric access systems.
When a face that is detected is not among the ones registered in the database, the system sends an alert to the security panel to take the appropriate action required. For a new user/employee, the system will help him/her register by capturing few photos in different directions with auditory feedback. Then the system prompts the new entrant to speak his/her name out loud. Then the system will accept his/her name after UID verifications and save the details of the new employee in the central server. From next time onwards, the system will recognize this user as a registered one. Likewise other standard attendance management system, our system will also note down the employees’ in and out time along with the absence and presence as well.
But this is just the tip of the iceberg of what we have envisioned to design and implement in the office premises. The behavior and Sentiment analysis (which has been explained above) can be used with the face detection based recognition attendance management system to monitor employee’s erratic and unusual activities during office hours and analyze the behavior of the employees within the office in real time to know the malicious intention of unscrupulous people, and finally send those data to cloud for further analysis. Although many offices have security cameras installed, their function is rarely anything more than recording video footage. These cameras can be used to implement computer vision in office settings. Bringing computer vision into corporate offices could serve as a solution to identifying the mental health state of employees. Face detection, face recognition, and emotion analysis are some of the many features of computer vision and would help in addressing the issue of office security.
Constituents & technology:
The system consists of a camera, a fingerprint sensor, proximity sensor, a processor (SBC like Raspberry PI where Intel optimized python is used), touchscreen with a microphone and a speaker. On the software side, it will consist of a Desktop software for managing the device data and/or mobile application for viewing and editing of data/details of the Iogs/IDs. Tensor flow, OpenCV, Keras, Google spreadsheet.
Image Processing and Feature Extraction:
Image Processing and Feature Extraction process
Deep learning model for face embedding:
Hardwares:
Detailed step of implementing the work using Jupyter Notebook on Intel DevCloud:
You can connect with Intel DevCloud using either a terminal or Jupyter Notebook. I have opted for the latter option and signed in with my credentials. Once you click on Jupyter option, the system asks you to provide the URL containing a “UUID” argument provided in your invitation mail (Fig:1). After clicking the submit button the screen looks something like this (Fig 2):
Fig 1 & 2
Now you can have the one-click login option to Jupyter with your ID and password. The Jupyter notebook repository opens where you can create your .ipynb project. I have created the entire project here using the TensorFlow, Keras, pyAudio, speech recognition and OpenCV modules of python. Some of the screenshots of how I have built the project are as follows:
Fig 3 &4: Importing various python modules in Jupyter Notebook
Fig 5,6 &7: Jupyter Notebook on Intel Dev cloud
The application works something like this. It opens with a screen (as shown in Fig below) where registered user/employee can check in just by pressing the check-in button. The camera turns on and recognizes the valid employee and takes down his/her in time.
Fig 8: The GUI of the Face Detection based Attendance Monitoring
In case of a new registration, the system recognizes the user as Unknown and a voice command instructs the user to complete the registration process with his/her credentials.
Fig 9: The response of the GUI in case of unregistered/new user
Fig 10: The registration process of unregistered/new user
Once the registration is finished, the voice command will tell the user to wait for some time and capture a few photos. Internally the face embedding part will be done during this period and within a few minutes, the embedding and saving of new user’s data will be done on DevCloud which will generate a pickle file.
Fig 10: The face embedding process of unregistered/new user
Henceforth, the user can check in just by pressing check in button which turns on the camera to capture and recognize the registered user with a unique employee ID.
Fig 11: The system recognizes the user with a unique employee ID
If the employee wants to go out in the middle of office time, the system can track his mid in and out time. The module is voice-enabled, which allows user to give voice command to check in, mid, mid out and check out.
As a temporary database, I have used a Google spreadsheet to save all the attendance data of the employees which runs at the backend and take necessary action at times. For example, if an employee spends more than a specified time at the outside of the office premises, then he/she can get an alert mail to notify the fact that. The behavior of all the employees have been monitored in real time and update the admin/HR officer at times.
Fig 12 &13: The google spreadsheet responses
The system calculates the total hours spent for each employee in the office and takes care of the attendance and behavior and will inform the admin at times. Weekly and monthly report will be generated automatically.
In future, the system will help the employees ensure a better work culture and environment,efficiency in a secure manner.
