What is the Nucleus-Hermès ?
The Nucleus-Hermès is a multimodal biometric glasses platform that captures the emotional and cognitive state of the wearer, or as we like to simplify it: its state of mind.
What is it, and how does it work?
The Nucleus-Hermès is based on the science of psychophysiology. Psycho- for psychology and physiology, which means how the human body works. In other words, it uses information (biosignals) about how the body works to infer the psychological state.
The Nucleus-Hermès uses four main biosignals:
- Electroencephalogram (EEG) measures the electric field generated by brain activity.
- Facial electromyography (fEMG) captures the electric field generated by facial muscles.
- Electrodermal activity (EDA) is the change of an electrical property of the skin, the conductance, which is related to sudation (sweat glands)
- Photoplethysmography (PPG) is a technique to measure blood volume under the skin using light, from which we can infer heartbeats.
The Nucleus-Hermès integrates all the apparatus required to measure and record these signals and can even perform real-time classification for biofeedback applications.
As a function of the configuration of the system, it also records contextual information such as participant position (indoor or outdoor) and integrates a point-of-view (POV) webcam.
Why is it revolutionary?
Many researchers in the field have historically focused on a single (unimodal) or a couple of biosignals to conduct human experiences studies. Those who ventured into multimodal (i.e. many biosignals at the same time) data gathering did it under strict lab conditions. This happened because the systems involved were costly and getting the participants ready was tedious (45 minutes set up time). It often required a desktop computer (or a laptop in a backpack) to collect the data.
The Nucleus-Hermès was designed from the beginning with flexibility and performance in mind. It can generally be set up in less than 5 minutes and is fully wearable. The biosignals collected were specifically selected to provide the best insights about emotional reactions — mainly reflected through facial expressions — and cognitive state — measured through EDA and EEG.
It is currently the only system on the market that uses the smart glasses form factor, which simplifies the fit for the user as it doesn’t need to contact under the hairy surface of the skin, in contrast to most EEG headsets.
Further, most technologies that depend on facial expressions to analyze emotional reactions use video-based analysis, which is impaired by face orientation, lighting and skin color — although IR camera manages to circumvent some of these issues. By using fEMG and being worn, the Nucleus-Hermes can monitor the user’s facial expressions without requiring the placement of cameras in the environment, which tends to interfere with the right to privacy of guests and visitors.
What is it used for?
Currently, the Nucleus-Hermès is being used for multiple applications:
- The headset is pretty spectacular and futuristic, and we accept this label. Consequently, we are often invited to showcase our system at business or public events, which gives us an excellent opportunity to share our passion, entertain people, and bring awareness about the role of emotions in our lives and the concept of human experiences. One of our experiential stunts consists in capturing, through the POV camera, the most intense moment of an experience lived by a participant and delivering what we call an emo)ment (emotional moment).
- Under the proper conditions, we use the system to conduct human experiences studies to understand better what drives satisfaction and how to improve entertainment productions and services. This guides experience creators who aim for excellence. Most of our results are shared through our Mindscape-Émotions white book — you can read the article about our Mindscape-Initiative here.
- In the future, we plan to miniaturize the system into smart glasses that can be worn all day to bring a better awareness of our emotions and how they drive our perspective on life. Hopefully, this will help people balance their state of mind to improve their general well-being. It might even help mental health clinicians treat their patients.
- Another interesting perspective is the production of personalized experiences. By combining our technology, which gathers information about the preferences and emotional history of someone, with an AI-generated content production pipeline, we anticipate a future where people can live and share personalized and very satisfying content on-demand. You can refer to our article on this topic to get the details of how this could work.
Are there any risks associated with this technology?
Like all technologies, they must be used conscientiously to bring out the positive aspects. For example, using the Nucleus-Hermès requires the wearer to be well informed about the information collected and to which intent. We abide by the rule of gathering informed consent during our experiences. Transparency is key.
As our system evolves toward general adoption — we’re still many years away — we want to ensure that the wearer always has control over where the data collected is going, who has access to it and what it will be used for. We believe that the benefits outweigh the risks if the users are well-informed and given the tools to control their data.
In a dystopian world, this kind of technology could monitor an individual’s state of mind against their will, be used to intentionally develop addictive lottery or gaming systems, or develop persuasive propaganda. But the same technology could just as well be used to defeat such ill-intended applications.
Consequently, we believe that it is by writing this kind of article and sharing our results that we can best ensure that this technology will be used for the greater good.