Biometrics Level Up VR And Provide The Next Leap Forward In Human/Computer Interaction
Virtual reality (VR)headsets have come a long way in the past few years. Bulky, heavy, low-performance hardware has been replaced by comfortable, high-powered headsets that are constantly evolving. The soon-to-be-launched HP Reverb G2 Omnicept headset is setting the standard for a new wave of VR by incorporating biometric sensors that will enable more adaptive and personalised experiences.
What are biometrics?
Biometrics is the measurement and statistical analysis of our unique physical and behavioural characteristics. Biometrics have been used in mainstream devices such as mobile phones since 2007, with simple fingerprint scanners being used to unlock devices.
Nowadays, we see much more advanced use of fingerprint, iris, and facial recognition technologies, such as Apple’s Face ID or Iris Scanner on the Samsung S8. Other examples of biometric identifiers are voice, typing cadence, physical movements, and DNA. As these are considered unique to the individual they can be used in combination to ensure greater accuracy of identification.
However, biometrics is not only about identification and security. For example, fitness wearables and health-tracking devices include biometric sensor technology allowing us to use our own body data to improve our performance and therefore fitness. This technology can also be applied to virtual reality. By monitoring things like the user’s heart rate, pulse, stress levels, even mood, and then feeding this back into the experience new possibilities for making ultra-immersive, uniquely tailored experiences are opened up.
What do biometrics mean for XR?
Some VR headsets, like the Vive Pro Eye, Varjo, and Neo 2 Eye, already have eye-tracking to measure where someone is looking or what they are interested in. Eye-tracking analytics provide valuable insights, for example, they can tell us what users are focusing on and which detail generates the most negative or most positive reaction.
On its own, this data allows us to understand key areas of focus and attention, and that influences how we design an experience. But if we go a step further and pair that attention data with a measure of a user’s reaction to what they are seeing — biometrics — we can understand the user’s response even more. The data that eye, heartbeat, and mouth sensors capture will enable developers to create adaptive and personalised experiences driven by the user’s natural physiological responses to the content.
For example in training situations, sensors can measure a user’s cognitive load (the amount of cognitive load involved in a task is the cognitive effort or amount of information processing required by a person to perform the task) giving businesses a better understanding of the trainee’s performance and ability to make decisions. In applications designed to improve wellbeing, real-time insights can help reduce stress and improve comfort. In medicine, biometrics can be used for biofeedback, condition assessment, and treatment personalisation.
This is not science fiction
HP Reverb G2 Omnicept launches this month and includes a state-of-the-art sensor system that takes information from the heart (PPG, pulse plethysmography), the pupil (pupillometry), looking behaviour (eye-tracking) and information from the HMD and controllers to gain insight into the user’s cognitive load — a very specific but very telling mental state. Every person has their own cognitive load or information processing capacity and it is fixed, limited, and varies from person to person. HP believes that understanding a user’s cognitive load will take training, wellbeing, creation, and collaboration to the next level.
HP’s first Omnicept VR software release includes an SDK (software developer kit) for VR developers with an AI inference engine to track mental effort while users complete a VR experience.
In addition to motion tracking, and cameras that follow mouth movements, the headset has cutting-edge optics, inside-out tracking, spatial 3D audio, integrated eye tracking from Swedish tech company Tobii, and improved controllers with natural gestures.
The headset has been years in development and signifies a leap forward in human/computer interaction. As more user data is collected, HP will continue to refine and broaden Omnicept’s scope to include emotions beyond cognitive load. In the future, it could be used to test consumer reactions to different advertising ideas or improve the productiveness of virtual workshops by capturing the nonverbal cues of participants.
Privacy
There is much scrutiny about the acceptable use of intimate and personal biometric data. There is concern that companies with access to the data will exploit users’ emotional vulnerabilities. Therefore, companies must commit to protecting the information collected by biometric technology and privacy must be a default on devices.
HP states on its product page for the Omnicept that no data is stored on the headset and that “privacy controls incorporated into the headset and via the VR application enable users to manage the headset sensors on or off.”
HP also states that it is not selling the data, instead, it is using it to improve product performance to “achieve the most accurate insights possible.” Users must give consent to have data collected and biometric data is not active if a user opts out via the headset or VR application or if the headset is not running. This solution is one that “meets or exceeds GDPR requirements.”
Other devices that use biometric data, such as Apple’s Face ID or Touch ID, transform your biometric outputs into a mathematical representation that gets encrypted and kept in a sort of “underground vault” in your phone that doesn’t get backed up to any cloud service.
As long as data privacy is top of mind for companies operating in this space (or any space for that matter) we can look forward to a new generation of immersive experiences that offer a level of personalisation and adaptability beyond anything we’ve seen before.
