“alya” a DIY modular personalized open-source smart mirror
Two months has passed since the eclipse open IoT challenge 4.0 started, and in this first blog we are so excited to talk about our idea “alya-smart-mirror”. I will talk about our goals, the hardware/software architectures and what we have achieved so far.
The idea
There are already many impressive open-source smart-mirror projects out there but we wanted to achieve even more. We were inspired by the MagicMirror project which is a modular smart-mirror with support for a wide range of 3rd-party modules that adds more features to the mirror such as showing the weather, reading the news, or controlling lights at home. “alya-smart-mirror” is another DIY open-source smart mirror that has voice/visual/gestures recognition at its core. The idea is to build a completely personalized smart mirror that configure itself automatically based on the user who is standing in-front of it. “alya-smart-mirror” is built in a flexible way to be modular and supports 3rd-party add-ons. We want also to go even more by supporting an easy to use hybrid app that can be used by the user to completely configure the mirror and all the add-ons.
The goals
Our overall goals are:
- Open-source and providing good documentation to explain all the steps needed to build and configure the mirror.
- Support for different visual/voice/gestures recognition options to make it more flexible for users based on the requirements with focus on the offline options to make it affordable.
- Modular and provides an easy way for contributors to develop, test and integrate the modules with “alya” core. All 3rd-party add-ons/modules should receives alerts about the user standing in-front of the mirror allowing them to develop personalized features. Similarly the add-ons can subscribe to optional gesture/sensor events.
- Provide good user experience by having a better accessibility, usability, and providing more efficient way for the daily interactions between the user and the mirror. For that we aim to provide a mobile app that the user can use to control and configure all the aspects of the mirror.
The components
An overview of the components in “alya-smart-mirror” are shown below:
- Mirror: is the main component which has all the hardware
- Backend: a server that provides all the APIs required by the mirror app/mobile app. The backend also interact with external cloud services such as amazon web services. The backend can be deployed either in the mirror or in the cloud.
- The mobile app: the user mobile app that will be used to trigger the face recognition training process and to configure all aspects of the mirror.
The hardware
We were lucky to be one of the 12 teams who won the starter kits. Therefore we had the chance to build something cool. We also received the UP squared IoT development kit as a gift from intel which gave us the possibility to do much more. An overview of our hardware stack is shown below:
- UP squared board: is the heart of the project where we run the mirror app/backend. The board is so powerful and makes perfect sense for our project since at the heart of “alya-smart-mirror”, we need to do intensive processing required for the offline voice/visual/gestures recognitions. We are also using xbox kinect 2 which can be connected only to limited boards since it requires USB 3 and faster CPUs.
- xbox Kinect 2: used for advance gestures/body tracking.
- Matrix Creator one: although in theory having the above is enough for our use cases (voice/visual/gesture recognition), we wanted to test the matrix creator which abstract the complexity of writing voice/visual recognition applications using their matrix OS. It also has a built-in sensors and support for zigbee/zwave protocols which makes it the perfect hardware for smart home applications. We are using it mainly for voice/face recognition, collecting sensor data, and controlling home appliances.
- Raspberry pi 3 model B: unfortunately we couldn’t make the “matrix creator one” works with the UP squared board since it is not compatible. Using the raspberry pi was the only option.
- Typical smart mirror hardware: this includes the monitor, two-way mirror and a first aid cabinet.
The software
An overview of the software stack is show below:
- Alya core: currently it is a react app that runs over electron as a desktop application. The core has the main functionality including voice/visual/gestures recognition.
- Alya component: this is an electron react component. It can be either a core or 3rd-party component.
- IoT device platform: we are using amazon IoT but any other platform can be used such as eclipse IoT, watson IoT, etc..
The progress
The progress so far is shown below:
- first version of alya core is completed.
- first version of voice/visual recognition using matrix creator.
- first version of the backend
- first version of the mobile app
- a POC add-ons including a digital/analog clocks, and a youtube add-on.
The challenges
So far we faced the below challenges:
- Making the matrix creator one works on UP board squared was a challenge. Seems that we would need to do some changes to make it work, since we don’t have enough time till the end of the competition, we just used a raspberry pi.
- Some issues during development when trying to run a react component in an elector react app. We found workarounds for these issues, we documented them in the readme file of the repos.
The name
“alya” is the name of one of the stars in a triple star system called “Theta Serpentis” which is 132 light years away from the Sun. The name has an Arabic origin and is a popular female name in some of the Arabic countries. The name sounds cool and this why we chose it :)
The team
We are a distributed team of 5 enthusiastic techies who resides on Germany, India, and Hungary. More details are below:
- Bilal - IBM - Munich
- Tareq - IBM - Böblingen
- Sara - Freelancer - Hyderabad
- Ibrahim - Master Student - Hyderabad
- Ali - Bachelor Student - Budapest
Links and resources
- All the code is open-source and can be found under this github organization.
