The Four Types of Augmented Reality

Have you thought about using Augmented Reality for your business? Well, first of all, you have to figure out whether or not such an experience will be a good fit for your purposes? There are plenty of good examples out there in the market, but unless your particular use case is very similar, it may be hard to know for sure.

Therefore I’ve categorised AR into four main types:

1. Augment — Your World
2. Augment — Your Self
3. Augment — Your Way
4. Augment — Your Things

Therein, you can identify which experience (if any) applies to your use case and ultimately if AR is right for you (will it work and will it bring results). Choosing the right one is ultimately a matter of tradeoffs between the following interdependent factors:

• Environment/Context: Augmented reality is a process in which digital content is superimposed on the user’s view of the real world. How much is known — or can be known — about this environment/context influences the optimal AR approach; including the selection of the most appropriate technology, for the particular use case.
• Use Case: The context in which the Augmented Reality application will be experienced by the user, and the desired result. For Eg an Augmented Reality web application as part of an e-commerce store, designed to enhance the shopping experience.
• Technology: There are a variety of technologies available to developers creating AR experiences, but selecting the right one — or even the right combination — depends on the particular use case and the user’s environment/context.

1. Augment — Your World

The user’s surrounding environment is augmented; be-it their living room in their home, the patio in their garden, or wherever they are experiencing the world at that moment — hence your world. An example might be in an e-commerce store, where you’ve found a dining room table you like and you want to see what it will look like in your dining room; or perhaps you are out & about and you want to play with your favourite Pokemon characters in the popular Pokemon go app.

Dining Room (digital content) table augmented into the user’s dining room.

Pokemon character augmented in your walk along the promenade with Pokemon Go

In these circumstances, the augmented reality experience can assume very little about the environment, which changes with wherever the user goes. Therefore the technology Six degrees of freedom (6DoF) is used, which utilises the hardware such as the gyroscope onboard the user’s device, to correctly orientate the digital content (which is augmented in the user’s view of the real world). In addition, computer vision is utilised, to detect surfaces and assist this orientation.

2. Augment — Your Self

Users augment themselves or other people — the physical human form. Be-it in selfie mode in Snapchat where they add a face filter to render themselves with a scary lion face; or perhaps they’ve found some spectacles or jewellery and they’d like to “try before they buy”; or even a doctor who’d like to get an x-ray impression of their patient’s arm.

snapChat face filter: user renders a bright bristly moustache complete with aviator glasses and throwback haircut.

have a go yourself at voala.io

Although these examples of AR, are from different industries, they are nevertheless dealing with the same subject matter (context) — the human form. And although the human body varies from person to person, there are many commonalities that important assumptions can be based upon; helping establish a high-level understanding of the human form, which is a fundamental part of the computer vision technology used in these examples.

For example in the jewellery shopping experience above from voala.io, the application’s computer vision algorithms, not only recognise the human hand but also the “ring finger” and the finger’s different parts; as to correctly augment the ring on the correct finger and in the right place!

3. Augment — Your Way

As a user embarks on a journey, digital information is superimposed upon their view in real-time to assist their navigation, from one destination to another. Aka “way-finding” or “navigation”. Google Maps Live View is a well-known example, where GPS is the primary technology used to locate the user in real-time; and then directional arrows are augmented upon their view to help them navigate their way (In addition 6DoF and computer vision orientate these directional arrows).

Google maps live view (source: techCrunch)

But what about where there is a lack of GPS coverage, for instance when you are in an indoor environment, such as a shopping mall, airport or hospital, where google maps can’t assist you? These environments are like mini-cities and need to be navigated all the same. Here, mapping technologies, such as point cloud maps, provide AR experiences with a complete 3D spatial map, for locating the user within the environment in real-time. ARWAY is one such platform, providing mapping technology and a portal for building the AR experience.

point cloud map courtesy of ARWAY

4. Augment — Your Things.

The user sees digital information superimposed on the view of whatever it is they are inspecting, to assist them in the process. A typical example is a support engineer, who has been tasked with fixing equipment/machinery. Aka “remote support”, “remote inspection”, “remote assistance” etc — in industry & enterprise. The efficiency savings that can be made, in serving machines through AR in this way are significant. As such, cutting edge AR glasses (eg the Holoens) are utlised and high fidelity digital twins are made (of the equipment) to capitalise on the technology and bring the greatest return on investment.

support engineer servicing equipment using AR (picture courtesy of Utility AR)

The environment/context, in this use case, is the equipment being augmented, which can take many forms and therefore is not subject to any universal assumptions. Without this intelligence, there is little that can assist the augmentation process. Therefore “digital twins” (3D digital replicas) of the equipment are created; that provide the AR experience with the necessary spatial awareness — to augment this equipment effectively. Creating these assets can be resource-intensive, but when considering the amount of equipment manufactured — that may need serving regularly — such a cost is justified. Visit Utility AR to find out more.

a Digital Twin prepared by a 3D specialist, for assisting Remote support (picture courtesy of Utility AR )

Now, Go-Forth and Augment –Your Reality

I hope by distilling Augmented Reality into 4 main concepts, I have given you a good starting point, for exploring the immersive medium. Once you’ve identified whether or not you’ll be augmenting: your world, your self, your way or your things; you can consider how you will do so, within the context of your use case — including the selection of the right technology etc. And as you weigh your options, you’ll realise the optimal approach is most likely a matter of tradeoffs — which is a reality of this emerging medium. For there are indeed limitations to AR, which are being addressed — with advancements in technology etc — but from where and how exactly, is the subject of my next article.