The Mixed Reality Revolution is here, and it’ll change your world forever
The past 30 years have set the stage for technological and cultural change at an unprecedented scale. The rise of Personal Computing, Internet as well as Smartphones have completely redefined the way we interact with our world as well as each other — today, over 99.9% of the world’s data is stored digitally and half of the world’s’ population is already part of the internet, a figure which is expected to reach closer to 100% by 2020. Every day we’ve been growing more reliant on modern technology as we collectively craft a separate digital universe of information byte by byte — a universe we use to manage everything in our lives from money to our closest social circles.
The digital world, however, holds little resemblance to our physical world — after all, the former is still governed by screens and 2D imagery. Companies have used their mastery of design to make interactions with the digital realm feel more natural through features such as slick user interfaces and touchscreen interaction, but there’s only so much that can be done in a rectangle. Currently, we interface with the digital world at a distance, looking through the windows of the devices we own, but never daring to step any further. The digital world is a world of information, while our physical reality is one of experiences — and because of that distinction those two realities have always remained separate, only connecting to each other through tangents.
But that’s bound to change. The recent rise of Virtual Reality has brought new ways of experiencing information into the light, inspiring a new wave of interaction design and experiential software that enjoys a true sense of presence in digital worlds. Now, a new form of computing stands on the horizon, creating a plane of reality that intersects the physical and virtual world that’s both exceptional and familiar. It is the result of the convergence of several advanced fields of technology that when put together will spur a new age of contextual computing, where digital information seamlessly blends into our physical reality as active parts of our environment — a world where computers can understand our surroundings and where technology feels unbound from the human experience.
This is Mixed Reality — it’s real, it’s here, and it’s one of the biggest technological races since the rise of personal computing.
Beginner’s guide: the Virtual, Augmented and Mixed Reality distinction
While Virtual, Augmented and Mixed Reality technologies are all similar in many aspects, they have some fundamental differences. Virtual Reality headsets like the Oculus Rift or the HTC Vive completely replace your current reality with a new one, putting you in a 3D generated world with little concern for your immediate surroundings. It invites you into its world, and it’s isolationist by design, which is one of its main strengths (fully immersive) as well as weaknesses (disconnect from your context).
Augmented reality, however, overlays digital information on top of your real world, the most notable example being the popular game Pokemon Go. And while it can make for some interesting applications, it is nothing but a rough digital overlay — it doesn’t truly understand your space, which stops you from making applications that truly use your world as a canvas.
Mixed Reality, however, is a completely different beast that combine several types of technology into one device — differently from AR and VR, MR devices are constantly scanning your room and gathering a 3D understanding of your surroundings, using that information to seamlessly place digital information within your space and interact with it, all of which can be viewed through transparent displays as you naturally interact with the result using your hands. Unlike Virtual Reality, MR doesn’t invite you to a completely different world — rather, it invites the digital world into yours.
The Race to Build a New Reality
The race for Mixed Reality is already happening, albeit mostly under shadows. All major tech companies like Google, Apple, Microsoft, Intel and others have been advocating resources to Mixed Reality ventures, with new players like the Meta and Magic Leap also entering the game, each with their own distinct vision of the future. And while most of the hardware projects conducted by these companies are currently secret, Microsoft’s HoloLens as well as Meta’s Headset have development kits of their own in the market today, offering a glimpse of the future that’s about to arrive.
The HoloLens has a sleek design and looks nothing like an unfinished prototype. In fact, it’s a completely functional untethered holographic computer, requiring no wires or any external power to run whatsoever. It possesses 3 distinct types of sensors and 5 environment understanding cameras working together to allow the device to understand 3D space and place holograms, and the resulting mixed reality is finally brought to life by the two high-pixel-density lenses that sit in front of the user’s eyes.
Through a simple “gaze and click” interface the user can do familiar tasks like browse the web as well as perform Skype calls in virtual screens you can scatter around your environment with no restraints of location and size— you can also play around with a range of novel Mixed Reality apps ranging from creative tools to Games that showcase some of the unique powers of the medium. The HoloLens adds a sense of permanence to digital objects that’s uncommon with technology — since it creates and stores a 3D mesh of all the physical environments you use it in, it remembers where you set up your holograms — so if you drop a couple of 3D models in the living room, some others in your bathroom and then leave your home for 3 days, they will all be exactly where you left them when you return. The only drawback to the HoloLens is its limited field of view — but it’s a temporary shortcoming that will surely be addressed in the consumer version. The amazing part is that all of its parts come together.
Another MR device that’s already out for developers is the Meta 2, currently being developed by a startup based in Silicon Valley. While this MR device is cheaper than the HoloLens (and possesses 90-degree field of view, compared to the HoloLen’s 50-degree FoV), it currently requires to be connected to a computer at all times, limiting your experiences to your immediate workspace. To handle interaction, Meta has opted for something less familiar in the realm of software but perhaps far more intuitive, focusing on accurate hand tracking that allows you to physically manipulate digital objects in your environment as you would in real life. The Meta development kit is set to release by the end of this year and Meron Gribetz, its founder, says their mixed-reality HMDs (head mounted displays) will be replacing all monitors in their office within the next few months.
What about the other tech giants? There’s very little we know about their unannounced projects other than the fact that they’re all being worked on. With all cases, interest in Virtual Reality and Mixed Reality seem to go hand-in-hand, as both are the hottest new immersive experience mediums in the industry. Google has been spending considerable amounts of money in their Virtual Reality department and is also reportedly developing a Mixed Reality device of their own. Intel has announced Project Alloy, a VR/MR headset hybrid set to release next year. Apple has several patents for Mixed and Virtual Reality headsets, both of which should see the light of day sometime in 2017/2019. Additionally, Facebook and Samsung have created entirely new departments with hundreds of employees dedicated to VR and related technologies, spending billions of dollars in the process. Even SnapChat could be tiptoeing their way into MR as well, with a steady stream of cash going into computer vision (which currently powers their popular Snapchat filters) and a new pair of glasses with 115-degree cameras (The Snapchat’s Spectacles), which could very well be a way to test out the market’s reception to wearables of this kind.
Either way, it’s no wonder all of these companies are paying attention — after all, these are technologies that could eventually replace all of our laptops, smartphones and computers simultaneously — causing disruption on a mass scale for all of the companies involved.
But out of all existing ventures, the most impressive one has to be Magic Leap’s, which may be on their way to becoming one of the biggest tech companies in the world. It has been backed by giants in the industry like Google and Qualcomm and is currently evaluated at $4.1 billion with no products of their own announced. Magic Leap’s goal is simple: they want to create a device that interfaces perfectly with the human brain, creating the transparent display to rule them all. They achieve that by going into a completely different direction than its current competitors, aiming to reproduce exactly what happens in real life: projecting light straight into the user’s retina through a digital light-field that’s indistinguishable from nature.
This means two things: firstly, that pixels are not visible. Because Magic Leap’s light-field is just light entering your eye like anything else in the world, it effectively spreads it evenly across your retina, allowing you to naturally focus on digital objects as you would in real life, pixel free. The resulting effect is so convincing that it fools the brain completely — for all intents and purposes, everything you see in Magic Leap’s displays is neurologically real. And if you take into account the rate at which these displays will progress in the future, it becomes hard to imagine a place in time 20 years down the road where digital objects aren’t any less crisp than the ones in the real world.
From Personal Devices To Personal Universes
If Mixed Reality becomes popularized, everything we know is bound to change — the eventual replacement of today’s computers and smartphones devices with technology of this type means that all digital interfaces that we know today will have to be rebuilt from the ground up. Overall, MR is expected to cause disruption on a vast scale, completely revolutionizing enterprise applications, education, social media and the $2 trillion dollar entertainment market — all while opening a door to a new way of being alive.
And this is important: Mixed Reality isn’t just a new type of technology — it’s an entirely new plane of existence. It’ll spur new ways of creating and consuming content, sure, but when you begin to speculate how MR will converge with other emerging technologies such as 5G and the Internet of Things, the possibilities become even more expansive. Users will be able to access all the information they need only when and where they’re needed, always presented perfectly on top of their real world instead of diverting you away from it. I tend to think that technology is only truly mature when it’s invisible to the user — in this case, MR is what technologically-mature computers look like, blending seamlessly into the human experience instead of detracting you from it.
“Mixed Reality devices are what technologically mature computers look like — invisible to the user, fitting seamlessly into the human experience”
Imagine educating yourself not in classrooms, but simply by exploring the world, learning about surrounding plants and animals by simply looking at them. Perhaps you would like to travel into the 1920’s for a day, replacing all of the surrounding cars, people and buildings with virtual retro counterparts. Or maybe you would like to be able to see through the walls of your office, just so you always know exactly where your co-workers are. What if you want to replace all advertisements that surround you with pieces of modern art? Or make that one bar always resemble the Mos Eisley Cantina from Star Wars, filled with digital alien creatures amidst the humans in the space. In a world of MR, there are no creative constraints, and this allows us users to have agency over their own realities in a way that we’ve never encountered before.
Ultimately, these devices will eventually give you the power to choose to live anywhere within a ‘vanilla’ and ‘fully virtual’ reality spectrum, representing a technological shift from a world of Personal Devices to a world of Personal Universes. Entire companies will rise and fall trying to create the perfect fantasy, and life may be bound to become the most complex choose-your-own-adventure game ever created.
The Fear Of The Unknown
Whenever there’s a new disruptive technology of this scale on the horizon, there is also the fear of the unknown. I’m sure some of the readers are terrified by the prospects of people “losing” themselves in a large scale “fabricated” reality, or by the implications that MR might bring for personal privacy, both legitimate concerns. I won’t get too much into the former, mostly because it’s a completely subjective and philosophical discussion. One could argue that the reality we already live in is being “fabricated” by our brains and MR just allows us to exercise control, much like we do over most areas of our daily lives. And while digital content in Mixed Reality devices may be ‘artificial’, the experiences and memories are still genuine. Regardless, there are no easy answers when it comes to this topic and it’s definitely an important discussion as we walk into this future.
There is one serious problem that needs to be talked about with MR, however, which is a direct continuation of something we face today: mass surveillance. Since these devices function primarily by acquiring a 3D understanding of our environment, we would be opening our most personal spaces to corporations and the government, leading to an unprecedented level of acquired knowledge about our lives without our full consent. As an added danger, as MR develops, these devices could also function as a gateway into our minds — after all, if you can tap into the reality of any single person, you can cleverly manipulate thought and behavior without being noticed, placing seeds of thought throughout one’s world almost in an Inception-esque manner. All of the companies involved in MR understand that trust and privacy are huge concerns and are certainly attempting to build products that address these fears, but we still need to be aware of the possible risks this technology may bring so the privacy discussion is always ongoing.
“If a smartphone is a surveillance device we voluntarily carry in our pocket, then MR will be a total surveillance state we voluntarily enter.” — Peter Wang
In the grand scheme of things, however, I believe that the potential of this technology is too great to pass and will likely bring much more prosperity than harm, similarly to all technologies that preceded it. It has the ability to reshape education completely, allowing an entirely new generation to learn not by looking at pictures at a book, but by seeing, living and vividly experiencing all topics they learn about. Other benign uses of MR can push people to be more empathetic, curious and free beings, dragging us away from our rectangular prisons and handing us a world far more beautiful when viewed with a new set of eyes.
Virtual and Mixed Reality bring about new forms of interacting with digital information that will revolutionize not only all the things we do, but also the way we perceive the world. It will open the doors to a new, unexplored reality filled with possibilities, where we will have the power to customize our human experience, away from the limitations and isolation of traditional computers and screens. It’s an unknown world —MR is the science of magic (and sorcery!) and the next few years will be vital to shape its future. If you haven’t gotten acquainted with VR and MR technologies, find a place that’s demoing some of the aforementioned devices and experience them as much as you can — they are not something you describe as much as something you live through, and that is the exact power of this medium.
“Ours is a journey of inner space. We are building the internet of presence and experience.” Rony Abovitz, founder of Magic Leap.
Within the next 4 years, we are bound to witness the beginning of a new reality and the rise of what may be a new trillion dollar market. This is not just the future of computers— this is the future of human perception.
Lucas Rizzotto is an Immersive Experience Designer, Artist and Developer. You can follow him on Instagram, Twitter, Facebook or contact him through his website.
Also, sign up to my mailing list so you can see more of my stuff (:
