It’s no secret that every large technology company out there is investing billions in developing a pair of augmented reality glasses. Industry experts estimate that we are two to five years away from seeing this new product arrive.
But when it finally arrives will it actually take off? Will millions of people voluntarily wear a pair of glasses?
Mobile phones started the revolution of bringing computing away from a fixed context to place a powerful computer in our pockets. Benedict Evans pointed out that we could think of a mobile phone as a powerful computer having capabilities that laptops or desktops didn’t — GPS, accelerometer, gyroscope, cellular connectivity, a camera and a multi touch interface. Those capabilities birthed multiple new industries and changed the way we communicate.
For augmented reality to become a powerful new platform it will need to do the same. It it will need to do more than just augment or add to our reality, it will need to enhance it.
When most of us imagine augmented reality we tend to think about animated 3D graphics being embedded in our immediate context, but it’s so much more than that. It’s about wearing a computer that knows what I’m looking at and knows more about my immediate surroundings than I do. It’s a computer that knows where I am in the world and knows what I care about.
As a thought experiment, let’s try to imagine what this new product will be good at. Let’s try to imagine why I’d want to wear a pair of glasses for hours on end every day.
We can start by thinking of the capabilities it will have. Here’s my wish list–
- A powerful machine vision engine that can recognize objects on the fly and match them to a real world database of objects.
- Eye tracking that allows the UI to understand what object I’m focusing on in my field of view.
- Enhanced location awareness that knows where you are and where you’re going.
- A powerful graphics engine that will allow virtual objects to interact with the real world, and allow for real time object occlusion (hiding a digital object behind a real object)
- A wearable haptic controller to allow more fine tuned interaction with objects.
- A capable voice assistant that will work in tandem with the visual interface.
Now how will this actually work? Will I be overwhelmed by the UI?
I read that when Spike Jonze was creating his seminal movie Her, he envisioned a world where technology was embedded, so pervasive, that it would fade into the background rather than taking center stage. He thought we would stop being so enamored by technology that we no longer felt a need to celebrate its presence everywhere. This feels right to me.
My guess is that the augmented reality interface that ends up winning will be much more minimal than we imagined. The operating system will probably use a similar model to the way the Apple Watch currently functions. Only one application will be able to run at any given time.
The operating system will be ruthless about only allowing a certain set of notifications to intrude on your current experience. With a user interface that takes over your field of vision, it will need to be even more careful about not distracting you unnecessarily. Notifications may be entirely haptic and auditory at first, with visual ones only being shown upon your request.
To continue the thought experiment let’s look at a few concrete use cases next.
Augmented reality can improve learning by making it more immersive and tangible.
There are many tasks that we do in our daily life that are difficult to acquire by reading an article or even watching a video.
Many of us have experienced trying to follow a recipe by watching a video playing on our phone or tablet — the screen quickly gets sticky and dirty as you keep switching back forth between food preparation and going to the next step in the recipe. If you’re am amateur cook like me, you keep looking for the correct measurement utensils.
Now imagine following the recipe when you’re wearing glasses that will provide visual hints directly on food items, and step by step instructions from your voice assistant. Remember the camera knows what you’re looking at, it has depth perception and the ability to draw. It could draw the digital tic marks on a glass so that the container can become a measuring cup. It can show you the difference between a teaspoon and tablespoon by showing you digital models of the exact size and shape that real ones would have.
By the way, you can still have a digital screen with a video playing from your favorite cook, that you can manipulate and move without it becoming sticky.
Tasks like cooking that require us to identify an unfamiliar object in a collection of objects then combine and assemble them in a specific order can be exceedingly difficult for a beginner. Food preparation, assembling a piece of furniture, installing an appliance, repairing a mechanical item like a bicycle or a car all fall under this category of tasks. The types of learning experiences that I described could all be improved with an augmented reality interface.
Augmented reality can improve our knowledge of our environment and our ability to navigate through the world.
I walk a lot and I ride a motorcycle, for both of those use cases the smartphone leaves a lot to be desired. It’s awkward to open up the phone to navigate, the Apple Watch implementation is closer to the experience I’m after with its periodic notifications that nudge me in the right direction.
An AR version of Maps could overlay better signage that will guide me directly where I want to go, without me having to look down or away at a key moment and lose my turn.
Audio is an important part of augmented reality. AirPods will work in conjunction with my glasses to give me a better understanding of my environment. This interface can point out things that may interest us, or help us find objects in our environment that we are searching for. For example when I walk past Whole Foods, I can ask my interface whether they have fresh yeast in stock, and it will find out, and even help guide me to the exact aisle in the supermarket so I can make my cardamom rolls.
Augmented reality will get its powers from knowing where we are in the world and helping us understand our immediate surroundings. I would prefer people interacting with the world than walking through it staring down at their phone.
Augmented reality can enhance our communication and enable us to better collaborate on complex tasks.
A lot of modern work is unsatisfying because many of us are tethered to our screens, we’re forced to sit down rather than move around and use our physical body. I have excess energy and start to feel nervous and jumpy when I’m forced to sit for extended periods. As a creator, I find our current digital tools amazingly powerful but stifling in how little I use my body when I create with them.
Currently when we have an extended discussion about a project or demonstration people sit their desks trying to follow the presenters mouse trying to stifle a yawn. Now if we are all wearing augmented reality glasses we can all see the same content rendered on a virtual white board that we can walk around and discuss. We won’t need a projector or large screen television. We can manipulate digital objects such as architectural models and product mockups together, change their color, materials and scale.
When I speak while wearing augmented reality glasses I can summon up supporting visuals that complement my language. Augmented reality will help us share what we’ve created with others in a whole new way, and it should make working together seamless and more fun.
Gaming & Entertainment
Augmented reality can make gaming and entertainment more personal, and the every day is even weirder than it already is.
When Apple first unveiled their step towards an augmented reality platform, ARKit, I was in Tel Aviv on a work assignment. To say that I was excited about Apple releasing AR for a mass audience was an understatement — I was like a little boy, who was told that Christmas had come early and my presents were already in the room waiting for me to be unwrapped.
At night in my hotel room I downloaded Apple’s development kit and began to modify the examples they had provided. With a little effort I had something working — one of the characters I had created was hovering in the air in front of me. It was magical, something out of my imagination had become tangible and was sharing the room with me, jumping up and down on my hotel bed.
Gaming is one of the most natural use cases for AR — this is where I can enter into that neon world that we all so imagine when thinking about augmented reality. The whole world can become a natural game world, where game physics and fantastical environments are layered on top of our reality. The whole world just became a virtual game board.
There are so many questions and possibilities to explore. How does gaming change when it can enter my immediate context — my living room, my street corner and I can share that altered reality together with my nieces and nephews at the same time?
What happens when I can create a personal relationship with my favorite imaginary friend, when it knows me, and we go out on adventures in my backyard? Will it be there just to entertain me, or can the story world become an even more powerful tool for learning than it already is?
The Disney that we all know came to be when Walt Disney made hand drawn animation into a sophisticated art form by combining it with a camera. When computer graphics became sophisticated enough to animate and create visual worlds Pixar was born. When augmented reality combines with Ai the next large media company will emerge.
In this article, I’ve focused on the positive applications that AR technology enables. I’ve stayed away from making predictions about applications that give you more information about the people around you. These use cases have a slew of privacy implications and can quickly become downright nefarious. Before we begin to use AR and facial recognition to share information about each other, we need to develop a better model for how individuals will own, protect and share their data. But that’s a discussion for another day.
My hope is that from reading this article, I’ve got you thinking about AR and what we can do with it. Then take the next step – start building, experimenting, and tinkering so that we can build a more interesting future together.
Need help with your product?
Have you been thinking about what AR can do for your company? Are you doing interesting things with machine learning but having trouble articulating your product? Get in touch! I love helping passionate teams build new products.
Swing by and checkout what I’ve helped build in the past — an AI powered primary care app that’s been used by a million patients, an app that donates your money to improve the world and a tamagotchi that feeds off your step count to encourage you to stay fit.
Connect with me on LinkedIn.
Want to build a new game company that is AR native here in New York? Drop me a line. We’re fundraising for UMI.