Augmented Reality

Augmented Reality is an interesting and one of the most talked about technologies of the recent time.

Before we begin, let’s first understand what augmented reality is. In simple words, it is just a blend of physical world and the information world.

Remember when you are listening to a song with your high-bass headphones on? You are immersing yourself in an experience which is blend of physical perception (sound) and the information world (music). In other words, augmented reality is a mixture of direct and computer-mediated perception.

The computer-mediated perception is developed by AR designers whose role is to prepare actions which will be triggered by the user interactions when the AR experience is played.

What makes AR experience great is its real-time nature. Since the direct perception is experienced by humans is in real-time, the computer-mediated perceptions need to be in the real-time too. Any delay between them will create negative impact on the overall experience.

What properties and rules should an Augmented Reality experience abide by?

As it turns out, there are three key components to this:

• 1) it must combine virtual and real information, with the real world as the primary place of action;
• 2) it must be interactive with real-time updates;
• 3) it must have virtual information registered in 3D space, in the physical environment.

In essence, while moving in a physical environment, the user’s actions (movement/interaction) have direct consequences on the rendered virtual elements, and all of this happens in real-time. If you pay close attention, these rules do not discriminate a specific output device or interaction medium, meaning that, in theory, AR is not limited to displays and visual stimuli. 🤔However, audio, haptic, olfactory and gustatory AR is more difficult to achieve.

Showcasing AR

Chances are, you’ve probably benefited from AR in your day-to-day life without even noticing it. For instance, have you ever seen or used one of these parking assistants? Or the use of AR in sports?

Yup, by definition, they are a perfectly good and simple example of augmented reality: the computer-generated elements help you gauge distances and trajectories in real-time! I know, I know, self-driving and self-parking cars are all the rage now, and they’ll pretty much make this example useless in the coming years, but I bet you never once thought of these displays as an awesome piece of AR.

Augmented reality has also seeped into gaming, and unless you have been living under a rock, you probably heard of Pokemon Go. Each year, more and more AR and VR games are added to app stores; especially AR-based games, as they don’t usually require additional expensive hardware to be purchased.

Existing Tools and Frameworks

The following are the current industry standards for building and testing out Augmented Reality apps.

ARTOOLKIT

ARToolKit was the first widely available and open-source library focused on AR. In order to create strong augmented reality, it uses video tracking capabilities that calculate the real camera position and orientation relative to square physical markers or natural features markers in real time. Once the real camera position is known, a virtual camera can be positioned at the same point and 3D computer graphics models can be drawn, exactly overlaid on the real marker. So ARToolKit solves two of the key problems in Augmented Reality: viewpoint tracking and virtual object interaction. Moreover, ARToolKit is available for the most common platforms: Windows, macOS, Linux, iOS and Android.

ARKit4

ARKit 4 on iPadOS introduces a brand-new Depth API, creating a new way to access the detailed depth information gathered by the LiDAR Scanner on iPad Pro. Location Anchoring leverages the higher resolution data in Apple Maps to place AR experiences at a specific point in the world in your iPhone and iPad apps. And support for face tracking extends to all devices with the Apple Neural Engine and a front-facing camera, so even more users can experience the joy of AR in photos and videos.

ARCore

Google has also developed a platform for building augmented reality experiences for Android and iOS. Using different APIs, ARCore enables your phone to sense its environment, understand the world and interact with information. Some of the APIs are available across Android and iOS to enable shared AR experiences. ARCore uses your phone’s camera for motion tracking, allowing it to understand and track its position relative to the world. Afterwards, it uses environmental understanding to detect the size and location of all type of surfaces: horizontal, vertical and angled surfaces like the ground, a coffee table or walls. Finally, it performs light estimation to estimate the environment’s current lighting conditions.

Honorable mentions go out to Vuforia and Wikitude as sensible alternatives to ARToolKit. There are plenty of alternatives out there, so take a look for yourself 😅

Wrapping up 🏁

I hope this post has inspired you to look into AR or has taught you something new. There’s so much more that I’d love to discuss and share regarding the current and future applications of AR, but by now I’ve probably exhausted your attention, so let’s leave it for another time.

In the next article, I will try to explain the architecture of augmented-reality.