What is reality? Most people would agree that reality, in its most basic sense, is the noun form of real. However, that begs the question: what is “real”? What makes the data our senses pick up valid? Even if our senses are reliable, how do we know the input is real? If our senses are intact, but are taking in false information, our perception of the world would then be false.
This debate has spanned for centuries, from the Scientific Revolution, with the philosopher René Descartes’ “I think, therefore I am” to today with conversations on whether or not we exist within a simulation. However, we as a species have advanced to the point where we can create reality, and not just in a philosophical sense, but in a real way.
I’m talking about virtual and augmented reality and these technologies will literally change your perception of everything.
Virtual Reality: Virtually Real
Changing reality sounds like something only a fictional character could do. While with current levels of technology we aren’t breaking space time, we are altering how people see the world in two main ways: virtual and augmented reality.
Virtual reality is arguably the more immersive version of the two technologies. It places the user into a 3D simulation, one only they can see, usually (but not always) using a headset to achieve this effect. It’s most widely known and used in the gaming sphere, however it has been used in the realm of military training, specifically with Airforce pilots.
The main purpose or gimmick of VR is to essentially trick your brain into thinking you are somewhere you are not. Games like Superhot and CREED: Rise to Glory are great examples of the immersion you can only truly achieve with virtual reality. It’s all an illusion that gets more and more real as technology advances. Virtual reality engineers are like magicians and virtual reality is their magic trick.
However, for the sake of education, I’m going to spill the beans on how it works. The primary challenge in achieving true immersive technology is to imitate how we perceive reality in order to make it seem as believable and natural as possible. To do that, VR (shorthand for virtual reality) headsets use eye trackers to try and predict the user’s line of sight.
The headset reflects infrared light off of the surface of the viewer’s eye. Headsets also have software that sets landmarks in the eye, such as the pupil and iris. Coupled with software that records the change over time of those reflections, the headset can track where the viewer is looking.
The technology is similar to graphing. When graphing on a coordinate plane, the points you use are like the landmarks or points of reference eye tracking employs. The lines you draw between those points act as a map. The more points you put on a graph, the more complex and accurate the image is. You also have an easier time distinguishing what the lines are supposed to form.
You may be wondering: why is it so important to track a person’s eye movements? Can’t you achieve the same effect by tracking head movements instead? In reality (unintentional pun), tracking head movements rather than those of the eye was how virtual reality worked a few years ago, however technology has strived to be more convincing and is constantly improving. Remember: the point is to be as persuasive as possible.
But here, we run into a problem: there is more to vision than simply where your eyes are pointed. Seeing what you are looking at is the most relevant application for sight. However, what about seeing what is not directly in front of you?
Experiment time: stare at the device you’re using to view this article, whether it be a phone, tablet, laptop, etc. When I say stare, I mean stare. Don’t pay attention to anything else for a few seconds.
Now, put your hand in front of your face and move your device back, focusing on your hand. Notice how you cannot see everything else with perfect clarity in either scenario? Your focus changed, so the level of detail in your environment changed. You can really only make out movement instead of the detail you are used to; that is how our species has adapted to keep ourselves safe. You don’t really need to see the texture of the fur of a half ton animal charging at you- all you need to know is its a bear you should probably try to avoid.
What modern virtual reality tries to mimic is that focusing effect: the headset sees what you are focusing on and blurs the rest of the simulation, just like your eyes do in real life. Eye tracking coupled with head movement adjustment is about fifty percent of the equation.
The other fifty percent is making sure you can actually see something. Just like game developers, VR engineers create a 3D world for the consumer to “play” in. However, the trick is camera work: they use the headset as the “camera” of the game. The difference between a VR simulation and a normal 3D video game is, at its core, literally just camera work.
For reference, the camera in any game is the viewpoint through which the game is played. It’s like the three basic points of view of a story; you have your personal first person, occasional second person, and popular third person. The headset puts you into the driver’s seat and the simulation is a first person only experience (so far).
Finally, the last steps are equating head movement to motion controls (similar to using tilt controls with a controller), any additional user inputs via touch controllers, and you have yourself a functioning virtual reality headset.
Outside of the Simulation
The most advanced headset we’ve seen belongs to the company Oculus and their Oculus Quest headset. That headset aims to achieve what virtually (another pun) no other headset of its kind has achieved, which is the coveted all-in-one status. However, it too has failed to fully realize the sheer amount of potential virtual reality has.
Virtual reality can be applied in almost any field that requires training.
I’m serious: virtual reality’s applications are therefore limitless in any work environment. From military training to surgery practice to civil engineering, virtual reality can be used in basically every aspect of any job. This technology can not only be used to save corporations potentially millions of dollars in unforeseen flaws- it can also save lives.
Take my civil design example: if a part of the design goes wrong in the simulation, engineers can change their designs to accommodate for the foreseen failure. This way, a multi-million dollar mistake can turn into, at most, frustration with varying degrees.
In fact, I challenge you to think of a single profession that could not use virtual reality under any circumstance.
I’ll be here all night…
Augmented Reality: Making Reality Better
Then there is the other approach to simulating reality: instead of rendering an entire world, why not build on top of the world we already have? This is where augmented reality is born and is why augmented reality is seeing more applications than virtual reality.
AR (augmented reality) is far less involved per se than virtual reality is. In its simplest sense, AR takes an image and pastes another form of media over it. For example, Vuzix has an app that translates text the user is looking at by literally putting translations over the original text. The point of augmented reality is to make our current reality better and more accessible, instead of simulating another reality entirely.
Take North Focals, the Canadian startup as an example. The glasses are paired with a ring that allows a user to add additional inputs outside of what they are seeing. The ring gives the glasses different functions, such as calling an Uber, telling you the weather, interacting with Alexa, and giving you directions. Wearing the glasses gives the impression of having a HUD display in front of you, however that is not exactly the case. AR is just as deceptive as VR and there is more to it than just a projection.
Around the right hinges of the glasses, there is a laser that points directly into the lens. That laser then bounces off a photopolymer material on the lens, creating a projection that then hits your eye at a 15-degree angle. That specific angle places the projection inside the lens in a 300x300 pixel area. The technology is nearly seamless; from an outsider’s point of view, the user seems to be wearing normal glasses, unless they look at the correct angle to see the little circle of the photopolymer material. It augments the mundane nature of wearing glasses and makes the use of glasses extend past simply correcting vision.
That vision (this is the last one I swear) is what gives AR the edge over VR. To be clear: neither is flat out better than the other. VR has the edge in gaming and professional training. AR has the edge in making life easier for a person and enhancing their experience. This augmentation can come in almost any field and place in life, similar to VR.
Like virtual reality, the presence of AR can also be felt in the gaming community, especially in the mobile market. In 2016, mobile users were introduced to the phenomenon that was Pokemon Go, a game that “created” virtual Pokemon the player could catch. While the game did eventually die out, the technology behind it was not abandoned and games like Wizards Unite and Minecraft Earth followed after.
AR is obviously not limited to smartphones, however the ability to use AR without putting a headset on gives AR more real-world uses than VR. Not to say VR has no future; that would be a blatant lie. Simply put, AR can be used in more areas of life than VR can because it doesn’t take you out of our world.
Virtual Reality (VR): A technology that simulates a 3D world for the user. This method of changing reality is usually done with a headset. This technology has seen the most progress in the gaming community, but its influence can still be felt in other areas of a person’s life. Examples include healthcare, education, work training, business collaboration, etc.
Augmented Reality (AR): The sibling of VR that enhances the real world, instead of creating an entirely different one. The ability to stay in reality and still have an enhanced experience puts this technology slightly ahead of its counterpart. Basically, there are more opportunities to use AR than there are to use VR.
The future of VR and AR is bright. When I say bright, I mean looking straight into the Sun bright. They have so many applications that will change the way we work, play, and live. AR could be paired with the Internet of Things and BCIs: a user could interact with different objects in the IoT by using a mind-controlled, AR interface (more on the future relationship of the IoT and BCIs in a future article). VR could be used to run simulations for basically anything and put the user into any scenario imaginable.
These relationships are just from the top of my head; the main strength of AR and VR is how versatile they are. These technologies can be used for so many things that it actually hurts to think about it.