Apple’s Next Big Thing Is…

Mixed Reality

Bruce Ironhardt
Oct 6, 2020 · 10 min read
Image for post
Image for post
IDG Insider Pro

Apple was once the most innovative company on the planet. They didn't just produce better tech, they created whole new product categories.

They did it with their PC business, then again with the iPod, then again with the iPhone, then again with the iPad, and then again with the Apple Watch.

But ever since Steve Jobs left us, Apple has been…honestly kinda boring. Year after year we’ve seen incremental updates to the same old products.

Don’t get me wrong, they’re perfecting their current product line and they put a lot of work in to create a coherent and frankly useful ecosystem connecting all their products. But this isn’t the Apple we all remember, this new Apple is one that would rather wait for a technology to mature before using it then to be the ones to the first innovate on it (except for maybe FaceID and AirPods).

So the question is… Will, we ever see Apple innovate something new again? And not just some new biometric or a pair of headphones. Will we ever see another iPhone like device?

And an even more important question… if yes what kind of product can we expect?

The answer is a definite yes and thankfully and we already have rumors on what it will be officially named.

Apple Glass.

Alright, the name isn’t super hype but I actually think this could be the next generation of how we interact with software.

For those that don’t already know, Apple will be releasing (at some point) a wearable headset that would allow users to experience Augmented Reality (AR) on a pair of “Smart glasses” instead of on a smartphone or tablet, think of games like Pokemon Go where developers can mix virtual objects into your surrounding environment.

Image for post
Image for post
Pokemon YouTube Channel

The reason why AR is so cool is that it can fundamentally change how we use software. Instead of having to interact with the constraints of a phone screen or computer screen, we can now interact with software as if it were part of the real world. I mean we aren’t too far away from a world where we can change the color of our car or add virtual screens to our home just by using our phones and AR. Obviously, the tech is still in its infancy but the possibilities are far-reaching and the big tech giants know this.

Just for context, basically all of them are working hard on the technology. Facebook, Microsoft, and Google are all investing heavily in the space and it seems like they’re all trying, at least right now to develop the technology in such a way that it will be THE next big platform.

Apple though might be trying to go for a slightly different and maybe more achievable approach, at least early on.

The Hardware of The Apple Glass

This is going to be the hardest part for the team at Apple.

If this is the next big thing and not some gimmick or prototype then the glasses are going to have to:

  • Look good for both men and women
  • Comfortably fit a variety of face shapes (meaning adjustable by either the users or Apple themselves)
  • Be relatively lightweight to avoid fatigue.
  • And be durable

We’ve seen from products like Microsoft’s HoloLens that in order to house all the technology needed to run the AR experience, the headset requires A LOT of stuff, making it bulky and not ideal for a walk outside.

Image for post
Image for post
Haptic AI

Now according to rumors courtesy of Jon Prosser (see his video here), we can expect that “they’re meant to LOOK like (normal) glasses, not intimidating tech” with some saying that they may be similar to what Tim Cook already wears or to a pair of Ray-Bans glasses.

So how was Apple able to achieve such a low profile?

Well since we haven’t seen the official product we can't be sure just how much they actually resemble normal glasses but assuming they do a recent patent filing might help explain it.

This filing refers to a wireless technology that would allow the iPhone to connect to glasses (similar to the original Apple Watch). This means that the computation can be off-loaded to the iPhone instead of being done on the glasses. Translating to huge weight savings.

On a side note, how cool would it be it they designed the glasses for a circular lens shape as an homage to Steve Jobs? That would be sooo cool.

Image for post
Image for post
Phone Arena

Another interesting leak from Jon was the lack of cameras on the glasses.

Now with AR, it would be expected to have forward-facing cameras to capture the world around you and superimpose virtual objects. But it’s believed that this was not added for privacy reasons.

We saw with Google Glass that people were uncomfortable talking to people that may possibly be recording them, so it seems Apple decided not to include them, at least for the first generation.

So… How are the glasses suppose to get the information required to impose images on the display?

The answer comes from the iPad Pro 2020 and the upcoming iPhone 12 Pro.

Namely the LiDAR scanner. LiDAR sensors can map an environment allowing it to both detect objects and get vital information regarding distance.

Image for post
Image for post
SPAR 3D

Additionally, the last important aspect of the physical device is of course the display.

The image needs to look nice and crisp, and from recent rumors, it looks like Apple is trying to go for 8k displays for each lens. If this ends up being true then that would be amazing, because it would allow for an ultra-smooth image.

The issue is that I’m seeing is if they’ll even be able to achieve that resolution because while it wouldn't be much harder to implement, it would be significantly more difficult to use.

The first issue will be the battery. How long will it be able to drive 2 8k displays for?

The next issue is processing. Will the iPhone be able to handle the processing for 8k displays?

An even bigger problem though is will the wireless bandwidth between the phone and glasses be enough to transfer that much data consistently without hiccups?

I of course have no idea what the answer is to any of these questions are but I’m definitely a little skeptical about whether Apple would go down the road of 8k, at least for the first generation. Of course, we’ll need to wait and see.

That all being said if the rumors are even half true then the glasses are looking to be real contenders and might be the first realistic option for consumers to enjoy an AR experience on a day-to-day basis.

It’s next biggest hurdle though will be the software.

The Software

We have a few leaks that I’ll go over but beyond that, the real issue I want to drive home is the importance of the user experience.

Now, this is Apple’s bread and butter.

When it comes to user experience, no one comes close to Apple’s attention to detail and effort. But I want to focus on some key areas that Apple will need to get just right if this product is going to be successful.

Interactions

How we interact with the software is just as important as the software itself. If it's clumsy and non-intuitive then it’ll fail before it even gets a chance to fly.

With glasses and AR, the first obvious kind of interaction that we think of is touch (or virtual touch?).

The only company I’ve seen do something like virtual touch really well is Microsoft with their HoloLens Headset. You can watch the video below to see there demo:

Being able to move, drag, enlarge, and shrink items in the virtual space will be key to making the device truly feel futuristic but also intuitive. I’m no expert but I think people’s first reaction will probably be to reach out to the virtual object and see what happens.

If implemented with virtual touch, it would open a new way to interact with virtual objects.

You’ll actually be able to toss a Pokeball at a new Pokemon or enlarge a virtual TV in your living room to fit the entire span of your wall, the possibilities with touch-based interactions are limitless but the issue will come from Apple’s approach to processing. Since all the processing is done on the phone it’ll be interesting to see if they're able to keep the interactions fluid.

The next kind of interaction will of course be a voice.

Being able to have Siri pull up your latest email, text message, or shopping list and pin it to the world could be hugely beneficial as a feature. And of course, I’m sure they’ll implement Siri to handle a host of other actions so I’m not as worried about how voice interaction will work.

The next possible method of interaction is through possibly the Apple Watch or even iPhone itself.

Sometimes the best form of interaction is with a kind of mouse pad and this is where being able to flick through lists like though your songs or photo albums could be more intuitive.

Also, it provides an alternative to virtual touch, so you dont look like a crazy person in the subway. Which is a plus in my books.

Look and Feel

The UI will need to be clean. Like extra clean.

People aren’t going to want non-essential information to take away from their field of view.

In fact, this could be dangerous if it does since it could cause potential accidents. Having the content be slightly off-center so that you can directly see what’s in front of you while still being able to have access to information will be key, while users will expect other content (like video) to be dead center.

It’s going to be interesting to see how Apple tackles this approach. I expect them to roll out a few updates and tinker these small UI elements as they get more user feedback.

Image for post
Image for post
A look at the UI from the Hololens from ARM

The actual assets used will also need to be appealing and crisp.

This is where Apple’s amazing UI design and the 8K displays should be more than adequate for most people.

It might also be too much to ask, especially for a first-gen product, but having the display be high refresh would make the experience feel like it fits in with the surrounding world.

People want an experience that makes their tasks easier to do, not harder so a lot the design choices are going to need to revolve around the design principle of removing friction for the user.

The Apple Glass will, I would guess, start out as a kind of heads-up display. Giving users access to useful information like maps arrows on the street when using maps or being notified of important content, like elevated heart rate from your Apple Watch.

As it begins to develop though and as 3rd Party developers get used to the new OS, we’ll start to see cooler and more futuristic software emerge.

Final Thoughts

Bringing it all together the device is rumored to be $499 USD.

This is honestly a pretty decent price for next-gen hardware from a company that considers themselves to sell luxury products.

I mean I’ve seen normal prescription glasses cost about as much, so if these end up being functionally good than many people may actually decide to purchase these instead. I did forget to mention but it's rumored that Apple will allow you add in your prescription (at an additional cost of course).

Beyond that, I’m glad Apple is coming out with something new and fresh and I really hope it does do well. It can end up being the benchmark for future AR glasses from other companies like Facebook, Google, and Microsoft when it comes to AR for the masses.

Hopefully, as time goes on and the tech beings to develop we see new use cases and more innovation on how this new kind of platform can be used to help people do things in new and unique ways.

I’d also like to say that I think people should temper their expectations since it is a first-gen product after all and so even though I mentioned a lot of cool possibilities, these are of course based on leaks.

That means that while Apple may be working on these features some of them might not make it to the first-generation headset.

Plus it's better to be surprised at how much better something is than you thought it would be. Instead of being underwhelmed how what you thought it was going to be.

I want people to be excited by AR but the development of new tech takes time.

Mac O’Clock

The best stories for Apple owners and enthusiasts

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store