Intel® RealSense™ Technology and the Future of Immersive Gaming
You’re in an unfamiliar kitchen, feeling mildly on-edge. As your pulse picks up, the cabinets start bleeding milk. It’s disturbing. Really disturbing. You feel yourself becoming more anxious and, as if by some kind of twisted black magic, the space around you reacts to your growing panic and soon you’re almost drowning.
This is just one of the nightmarish scenarios you might experience as a “Neuroprober” solving puzzles and exploring psychological trauma in the Flying Mollusk game Nevermind. “It’s very dark, it’s very surreal, and it’s very abstract,” founder and creative director Erin Reynolds says. It’s also a deeply immersive experience, thanks to biofeedback technology powered by Intel RealSense technology that uses a depth-sensing camera to seamlessly translate a player’s movements and visceral responses into the progression of the game itself.
That early appreciation for terror-lite gradually developed into a full-fledged love for the genre’s more extreme offerings. For Reynolds, the impact — mental, emotional and even physical — of even just seeing freaky films hinted at the kind of engagement that could be developed within the more inherently interactive experience of a first-person game.
In 2011, she began developing Nevermind as part of a grad school thesis project, with a concept that centered around fear and feedback. Reynolds was eager to integrate sensors for measuring physiciological functions not only to track how users were responding to the extreme situations in the game, but also to use that data to alter the action in real-time.
The successful Kickstarter campaign Reynolds launched to fund the early R&D included this bold statement: “If you let your fears get the best of you, the game becomes harder. If you’re able to calm yourself in the face of terror, the game will be more forgiving.”
And that gets at Reynolds’s commitment that Nevermind would have a positive impact on players. “It would be entertaining, but also benefit them in some way, shape, or form,” she says. Reynolds believed that by understanding their body’s response to unnerving scenarios and rewarding players for keeping their cool, they would then be able to better handle real life’s unexpected stressors and panic points.
At the time, the only consumer gear with the kind of monitoring capabilities she required were chest straps used to track heart rate during exercise. The system required gameplayers to lift their shirts up, rub a little conductive gel on their chests to ensure a clear signal, and strap on the device before starting the game. “The tech itself worked great for our purposes, but that is a lot to ask of a lot of players,” she says. So while it wasn’t entirely practical, the chest strap did provide enough of a proof of concept for Reynolds, whose timing and entry into the industry could not have been better.
“Between 2012, when I graduated, and 2013, wearable technology exploded,” Reynolds says. Driven by a growing public awareness and interest in wearable tech, the early concerns she heard that users wouldn’t want to have their vitals monitored entirely dissipated. It was a cultural sea change that coincided with huge technological strides.
And those breakthroughs included Intel’s development of Intel RealSense technology, including a depth-sensing camera and software enabling computers to recognize who we are, deduce how we are feeling, and interact with us in much more natural ways. Intel RealSense technology became the key to bring Reynolds’ dream, or rather nightmare, to life.
With the help of Chuck McFadden, a producer in Intel’s Perceptual Computing division, Reynolds was able to evolve beyond the bulky components and chest strap of her prototype gaming system. The Intel RealSense camera, a device the size of a stick of chewing gum, integrates two cameras, infrared lasers, and custom processing chips to capture the three-dimensional scope of a meatspace scene. Subtle gestures become immediate commands. “You can reach in and grab photos out of frames,” Reynolds says. “You can push doors open and push body bags away.” And yes, it keeps tabs on your heart rate just by looking at you.
For Intel, pursuing relationships with bright minds in the dynamic world of game design is a clear way to ignite innovation.
“At the highest level, if you look back at the history of computing and computers in general, most of the new technology that comes out was often led by gaming of some sort,” McFadden says. “Before, computers and games only understood you through the buttons you pressed. Now, with Intel RealSense technology, your games can begin to see you and react to you in ways that bring deeper immersion and meaning to your favorite pastime.”
Even if that pastime scares you half to death.
Photography by Kendrick Brinson
