The Spectrum of Reality
Yes, it’s a spectrum for now…
In the midst of building a mixed reality experience (an experience that blends the authentic and virtual realities and allows each to affect the other) we discovered several interesting questions:
•Where does unmodified reality end and virtual reality begin?
•How do each of the senses play into this spectrum?
•Why should we as designers care about this spectrum?
To help aid our efforts in understanding what we were building, we created the beginnings of a method for evaluating digitally enhanced realities or, as we began calling them, alternative realities.
More than just the intersection between the virtual and unmodified
We are not the first to dip our toes into defining this world, Professor Paul Milgram of the Mechanical & Industrial Engineering Department at the University of Toronto created the Reality-Virtuality Continuum. This spectrum defines the relationship between real and virtual objects and the spaces they inhabit. Some of the relationships that he defined that relate to the work we’re doing are depicted in the illustration below.
From everyday life, to “The Matrix” and everything in between
As designers, it’s our goal to create the best possible experience for the users and that means expanding the existing spectrum to offer a more easily digestible holistic view of an experience. Until this point, interaction designers have predominantly worked in two dimensions with experiences that generally only take place on screens. With alternative reality experiences currently in their infancy, now is a good time to start understanding how people will exist within them. Our first baby steps in cataloging and pulling insights from these experiences resulted in a definition of each of the five basic senses across the reality-virtuality spectrum.
We chose to use the real senses as a first pass to understand the human factors that govern alternate realities, as these experiences are still work to seamlessly immerse us. To complete this immersion, all of the senses must be convinced that what is virtual actually occupies the same space.
Above is our method for evaluating alternative reality experiences being applied to unmodified reality. As illustrated, all of the senses can be used at the same time in the reality we occupy everyday. Apply this same method to the popular science fiction film The Matrix, and each of the gauges moves to the far right of the spectrum.
The Matrix is representative of the perfectly virtual experience. All of the senses are manipulated and are thought to be real by the characters occupying that reality. This, of course, is science fiction and nothing currently exists that can fully immerse an individual to this level. Again, we applied this method to a different alternative reality, but one that actually exists at this time.
Tilt Brush by Google is a virtual reality experience for the HTC Vive that allows artists to paint in a 3-D environment, using the entire world as their canvas. While in this virtual world your senses of sight and sound are completely immersed, but a key component of the experience, touch, breaks the experience. This isn’t because the controls for the game are poor, it’s merely because the interaction of touch hasn’t achieved the same level of virtuality. All of the touch interactions work using the Vive controller, which works well as a controller, but doesn’t give the tactile response that working with a paint brush offers.
A virtual experience doesn’t need to be “The Matrix” to be great
The alignment of the senses is just one way to catalog and determine the successfulness of an immersive experience or alternative reality. There are many other factors that need to be included as this method expands and matures. Human factors beyond the physical such as social, cultural, and emotional must also be included for this method to truly be effective.
We hope to expand our thinking to these areas as we move into the final phases of our project. Stay tuned to see our progress.
Andrea Everman, Sarah Mitrano, Ian Morrow, and Daniel Park are interns at Moment in New York. Sarah is a recent graduate of Washington University with a BFA in communication design; Daniel is a junior at Parsons pursuing a BBA in strategic design and management; and Ian and Andrea are pursuing MDes degrees at the IIT Institute of Design. They’re all currently exploring the intersection of mixed reality and children’s education. You can follow the team’s progress this summer on Momentary Exploration.
UPDATE: The Moment summer 2016 intern project is live here.
