Redefining Reality: Dream with Your Eyes Open
Imagine you start your morning with a cup of coffee in a small shop in Paris. Then, you go out for a walk in Central Park in New York. You come back and start working in your office, which is located in Switzerland. After a long day at work, you fly to a beach in California to watch the sunset. Finally, you end your day at a 5-star sushi restaurant located in Japan. But most importantly, you did all of this from the comfort of your own home in Dhaka. For every millennial living in reality, all we seem to want is to escape from it! Ironic isn’t it? Don’t worry, XR has your back. This escape route from reality is made possible only through XR or Extended Reality- the next evolution in Human, Computer, and Environment interaction.
XR or Extended Reality is an umbrella term referring to all human-machine interactions generated by computer technology and wearables, where the ‘X’ represents a variable for any current or future spatial computing technologies and ‘R’ represents reality- meaning anything that can simulate a lifelike experience thus merging the boundaries of Reality and Virtual Reality. One of the most recent terms added to the list of technological advancements, XR has already gathered the interest of many engineers and scientists. So, what is XR? To know more about XR, first, we have to look into its constituents. The ‘X’ variable encompasses multiple technologies, namely- VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality).
How Virtual Reality came to being
Out of these 3 technologies, the one we are most familiar with is VR. Virtual Reality is the use of computer technology to simulate a fully immersive environment that can be similar to or completely different from the real world [1]. Standard systems use either VR headsets or multi-projected environments to generate realistic visuals and sounds. The VR interface offers a simulation of anything you can imagine in a 3D environment, instead of the conventional 2D screen.
VR was considered science fiction for at least a century, like in Stanley G. Weinbaum’s 1935 short story “Pygmalion’s Spectacles”. The story is about a scientist discovering a pair of glasses that can “make it so that you are in the story, you speak to the shadows, and the shadows reply. Instead of being on a screen, the story is all about you.”
Throughout the 1950s, VR was deemed as a white elephant because of its inability to be made cost-effective in the consumer end. Perhaps one of the earliest versions of VR is the Sensorama. In 1956 (patented in 1962) Morton Leonard Heilig (Currently touted as the father of VR) built the first Sensorama prototype. It was a large booth that could fit up to four people at a time. It combined multiple technologies to stimulate all of the senses. It had a wide-angle screen, and the participant had to pop their head inside a small opening of the box. The Sensorama was able to provide the illusion of reality by displaying stereoscopic 3-D images in a wide-angle view, body tilting and seat vibration, supplying stereo sound, wind to move the hair, and also had tracks for aromas to trigger the sense of certain smells during the film. Even after all the advancements in VR at present, some argue that the Sensorama is still an unforgettable experience. Some of the other primitive VRs in the ’90s was SEGA’s gamer VR, Nintendo’s Virtual boy, Vfx headgear, etc. These applications lacked the fully immersive experience and just cost too much which is why they ended up being major failures.
VR came into the spotlight when the earliest prototypes of the Oculus Rift showed up at the E3 videogame trade show in 2012. Thanks to the Oculus Rift gaming gear, the VR sensation was born. Oculus Rift was the brainchild of 22-year-old Palmer Luckey. As a home-schooled teenager in Southern California, Luckey spent much of his free time with electronics, fixing people’s iPhones and PlayStations. When he was 17 he started building the prototype in his parents’ garage, envisioning what a fantasy world might look like in a projected environment. After some crowdfunding and trying to make it in Silicon Valley, he finally made it. And boy did he make it big! The tech giant Mark Zuckerberg himself bought the Oculus Rift company for an eye-popping $2 billion in 2014. Palmer Lucy is still working towards his passion for virtual reality. In his words,
“Why shouldn’t people be able to teleport wherever they want?”
Oculus has almost single-handedly kickstarted the VR phenomenon. Soon a plethora of companies started announcing their own VR technologies. At present, there are at least 250 companies working on their VR technology including the tech giants Microsoft, Google, Facebook, Amazon, Samsung etc. Samsung teamed up with Oculus to launch the “Gear VR” which uses a VR compatible with the Samsung phones. At the top of the game is Sony, who integrated the VR with the PlayStation- which obviously became an instant hit. Gaming is the main industry propelling VR forward, but it can also be used for educational purposes, flight simulation, and military training. The most recent addition to the market of VR Gaming is the famous “half-life Alyx” which hit the market in March 2020 and brought VR to the mainstream gaming industry, inspiring nearly a million VR headset purchases after it released to rave reviews [2].
Augmented Reality or AR: Why it is NOT VR
In contrast to Virtual Reality, Augmented Reality is an interactive experience of a real-world environment where the real world is enhanced by computer-generated perceptual information across all sensory modulates (visual, auditory, haptic, somatosensory, and olfactory) [3]. AR projects augmentation on top of a real, physical environment. The main reason AR has been categorized as a separate technology from VR is that VR does not allow the user to interact with the real environment. Through AR, a person can manipulate and interact with his surroundings without actually altering the physical environment. Also, AR adds additive or destructive elements to the environment whereas VR replaces the real environment completely.
Although VR is relatively expensive and beyond the reach of most consumers, you might have already used the most common and primitive applications of AR. Ever used Snapchat filters? That’s Augmented Reality. It detects your face as the environment and augments different filters on top of your face. Can you think of anything else? Yes, the famous “Pokemon Go” is also a product of AR, which uses your real environment as the canvas for augmentation.
AR can be defined as a system that combines the real and virtual world, real-time interaction, and 3- dimensional registration of virtual and real objects. AR also has the scope of application in the military sector, where goggles can be used to measure the exact distance of the enemy and aim accordingly. It also has a major scope of daily-life uses, for example- holding your phone’s camera in front of a certain product to know its details, or holding it in front of a street to know the direction, or even faking an apple watch.
Merging AR and VR: Mixed Reality or MR
A letdown in AR is that you cannot directly manipulate the real objects, rather it works with virtual objects overlaid on the real environment. This is where MR or Mixed Reality comes in, where physical and virtual objects co-exist and interact in real-time. Mixed Reality is similar to Augmented Reality, but the key difference is that MR allows the virtual overlay to interact with the environment which gives a user of being a participant in the action, rather than being just a spectator. Unlike VR where the physical environment is shut out completely, Mixed Reality or MR encompasses the real world in combination with VR to create a new experience. The big dog in the field of MR is Microsoft, who unveiled the “Hololens” in 2016, the first fully self-contained holographic windows computer-glasses that enable you to interact with high definition holograms.
MR also has a massive scope of application in the medical field, where surgeons can practice and perfect their skills more effectively by using an interactive model of the human body, rather than performing on a patient directly. However, MR technology requires a significantly large amount of processing power and is insanely expensive.3D high definition graphic systems are too slow to present a fully immersive augmented reality environment. It can’t be commercialized just yet.
Microsoft launched the next generation “Hololens 2” that came out just last year, which expands the horizon of XR and provides a more immersive experience enhanced by the reliability, security, and scalability of the cloud and AI services from Microsoft. It has a resolution of 47 pixels per degree and a diagonal field of view of 52 degrees, improved hologram stability, accelerated eye-tracking and fully articulated hand tracking, new hardware, spatial audio, etc. Hololens platform will be able to use Unreal Engine 4 to develop applications. But this is still far from being commercialized, as a single pair can cost up to $350,000.
To summarize the difference between the technologies, it is presented in a tabular form:
So, all these technologies combined can give us a complete idea of Extended reality or XR. You might be wondering, what might be an example of XR? Well, like it was mentioned before, XR is more of an “umbrella term” used to market VR, AR, MR devices rather than being a technology on its own. It’s like how the term “Artificial Intelligence” includes Machine Learning, NPL, Deep learning etc., which are separate technologies on their own. XR has massive scopes of application, to name a few- Retail, Marketing, Real estate, Healthcare (Holographic surgery), Military, Education, and Engineering (Immersive Learning), Virtual events, Virtual merchandising, Training (simulating dangerous situations), etc. In fact, XR might just become a part of our daily lives, much like smartphones or PlayStations. After buying Oculus Rift, Mark Zuckerberg said-
“We are making a long term bet that immersive, virtual and augmented reality will become a part of people’s daily life”.
In May 2020, Qualcomm (who built the chip for the Oculus Quest VR headset) came out with a big announcement that they will be collaborating with 15 worldwide operators to deliver XR gadgets to consumers. These operators are planning to make XR a part of their offerings with 5G services. 5G will give extended reality a significant boost. Although many people believe that it will be 5G that finally makes XR sustainable, I suspect that it will require at least 6G to harness its capabilities thoroughly and accomplish anything truly remarkable. Nonetheless, extended reality will continue to advance as new technologies emerge. Many recently published papers theorizing 6G strongly suggests that XR will play a central role in our world in the 2030s and beyond. According to Visual Capitalist, XR is expected to grow 8-fold, reaching an estimated market size of $209 billion by 2022 [4].
XR has been gaining momentum only in recent years. In the coming years, holographic devices will become more immersive, and immersive devices will become more holographic. 20 years ago, scientists would have never believed we could fit terabytes of memory into a device the size of a stick of gum, yet here we are. In 20 more years, maybe we will be able to fit insanely high processing power into wearable XR glasses. The upcoming decade will blend the real and virtual world in such a way that it was only thought possible in science fiction. Soon enough, reality will be redefined.
References:
[1] https://en.wikipedia.org/wiki/Virtual_reality
[2] https://screenrant.com/half-life-alyx-developer-commentary-update/
[3] “The Lengthy History of Augmented Reality”. Huffington Post. 15 May 2016.
