“alya-smart-mirror” as sensor hub for your home

3 min readMar 15, 2018

Today is the big day for @eclipse open IoT challenge 4.0 competition, but we still have an opportunity to introduce new core module of Alya Smart Mirror . Sensor hub is one of alya’s features collects and shows user’s various sensor hub such as temperature, humidity, light, face detections, face demographics and much more.

Architecture

We are using the Matrix Creator which has around 8 built-in sensors and capability to detect faces and provides demographics information about the face such as emotions. We collect all these information and send it back to Alya. MATRIX OS has an out-of-the-box support for a powerful computer vision engine that can be used as a face detection sensor and list of possible sensor data. We have implemented an application that runs in the MATRIX OS and uses the computer vision engine. This app collects the information about detected sensor/face detection events and forward it to an addon on alya. The receiving addon in alya is asm-dashboard-addon, which can receive different data such as sensor data as well as voice/face/gestures recognition data. When a sensor events are received including face detection events, an MQTT message will be sent to the backend through Amazon IoT. The backend will send the message to Alya through socket notifications. The mirror will then load the data to the screen.

How it works

The matrix board sends events whenever it gets new sensor readings or changing in the data.

Sensor data will be sent to topic alya-data and payload like:

{
"dataType": "matrix-accelerometer",
"status": "success",
"data": {
         "x":-0.0260009765625,
         "y":-0.93707275390625,
         "z":0.37823486328125,
         "type":"accelerometer"
      }
}

Full list of possible sensor data is shown in the example below:

[ 
     {
         "x":-0.0260009765625,
         "y":-0.93707275390625,
         "z":0.37823486328125,
         "type":"accelerometer"
      },
      {
         "x":-0.001820000004954636,
         "y":0.4118800163269043,
         "z":-0.2692199945449829,
         "type":"magnetometer"
      },
      {
         "value":-2.5122909545898438,
         "type":"temperature"
      },
      {
         "value":95160.5,
         "type":"pressure"
      },
      {
         "yaw":89.37448120117188,
         "pitch":1.3995394706726074,
         "roll":-67.83077239990234,
         "x":2.071075439453125,
         "y":-0.43365478515625,
         "z":-0.530853271484375,
         "type":"gyroscope"
      },
      {
         "value":0,
         "risk":"Low",
         "type":"uv"
      },
      {
         "value":526.25,
         "type":"altitude"
      },
      {
         "value":29.052505493164062,
         "type":"humidity"
      }
 ]

The matrix board will send the sensor events based on the configured refresh interval inside configs/config,defaults.json. It also emits an event whenever a face is detected:

The matrix board sends also an event whenever a face is detected:

{
  "dataType": "matrix-face-detected",
  "status": "success",
  "data": {
    "location": {
      "x": 284,
      "y": 153,
      "width": 110,
      "height": 110
    },
    "tag": "FACE",
    "image": {},
    "trackId": 2,
    "demographics": {}
  }
}

Additionally Whenever a face’s demographics is detected, it will update the data as shown below:

"dataType": "matrix-face-demographics",
"status"": "success",
"data": {
{ 
"location": 
{ "x": 213, "y": 221, "width": 55, "height": 55 },
 "tag": "FACE",
 "trackId"": 2,
 "demographics":
  { 
    "gender": "MALE",
    "emotion": "CALM",
    "age": 25,
    "pose":
     { 
     "yaw": -0.24303536117076874,
     "roll": 0.04344254732131958,
     "pitch": -0.10279278457164764 },
    "face_id": "4"
     }
  }
}

Demo

When user is standing in front of alya it will detect his/her face along with gender, age and person emotion, also when the temperature or pressure are changing it will change in alya accordingly. Below an example when a person stands with different emotion.

Run

This app needs to be deployed in a Raspberry connected to a matrix creator one.

Please follow the steps here to create or deploy applications on Matrix OS:

Creating, deploying and running applications.

Prerequisites :

Installation & Registration

Then deploy the app :

git clone https://github.com/alronz/alya-smart-mirror.git
cd sensor-data
matrix deploy matrix-sensor-data

You can find more information about how to run the app in here

“alya-smart-mirror” as sensor hub for your home

Architecture

How it works

Demo

Run

More readings

Written by Ibrahim Senan