Offline face recognition for “alya-smart-mirror”
Almost a week is left for eclipse open IoT challenge 4.0 to finish, and we want to introduce the offline visual recognition capabilities of alya-smart-mirror(ASM). Having a visual recognition support in ASM is part of the core services which is used to recognize the user sitting in front of the mirror and loads his/her configurations. This is the most important core feature of ASM since it makes the mirror personalized. We selected an offline visual recognition service since it provides better privacy, easy to configure, and of-course less expensive.
Architecture
We have used the Matrix Creator offline visual recognition capabilities to train, detect and recognize users’ faces. MATRIX OS has an out-of-the-box support for a powerful computer vision engine that can be used for face detection/recognition. We have implemented an application that runs in the MATRIX OS and uses the computer vision engine. This app can be used to train the face for the first time, then recognize it later. The results of the recognition/training will be sent to the backend through MQTT message using amazon IoT service.
How it works
There are two phases for visual recognition. First is the training phase which happens when the user configure the mirror for the first time. Then the recognition phase which happens when the user stands in front of the mirror. Below I explain how both phases works:
Training phase
User will use the mobile app to configure his mirror for the first time. The mirror can be used by more than one user which allows the mirror to give a personalized experience for each user whenever they stand in front of it. The first time the user login into the app, he can train his face so the mirror can recognize him whenever he stands in front of it and loads his/her configurations. Once the user starts the training phase from the mobile app, he needs to stands in front of the mirror as explained below.
Recognition phase
Recognition phase is simple, the user needs just to stand in front of the mirror. Face will be detected by the visual recognition core, which will trigger the recognition process. When face is recognized, an MQTT message will be sent to the backend through Amazon IoT. The backend will fetch the user configurations and send them to the mirror using socket notifications. The mirror will then load all the configurations related to the current recognized user to provide a personalized experience.
Running the app
As explained, the visual recognition core in this case is an app that runs in MATRIX OS. To deploy the app, run the following command:
git clone https://github.com/alya-mirror/alya-smart-mirror.git
cd visual-recognition
matrix deploy matrix-face-recognitionMore details about the steps are here.
