Design and Develop For AR & MR
Are we ready For AR / MR and Spatial Computing ?
With the rise of Augmented-Reality SDKs (Apple’s ARKit & Google’s ARCore), the new Mixed-Reality Glasses from Microsoft HoloLens, and Magic-Leap, it’s time to step-into an entirely new world of apps and experiences.
And for the UX/UI Designers and the Developers among us, it’s also a great opportunity to create productive apps for the Spatial Computing! So if Cross-Platform devices (Mobile, Tablets, Desktop, and IoT) is not enough for you, Then maybe you should dive into the future!
Step into Augmented-Reality (AR)
Just like many creative developers and designers out there, my first experience with AR was back in 2009–2010 when AR was so primitive, we had to use some sort of markers in order to make it work…
And then in 2013, when I tried Google Glass for the first time, I was so excited because it looked like the beginning of AR as it should be — on your eyes (as Glasses), and not in your hands (like a Smartphone). But it wasn’t perfect yet, although I could see the potential in such product for contextual HUD when driving, riding a bike, or when navigating the city on foot. So It wasn’t true AR/MR experience, at least not for me…
From AR to Mixed-Reality (MR)
Few years later, when I joined Microsoft and tried The HoloLens, I was amazed, and I was also convinced that it’s the future of AR! For me, it was the leap from AR to MR. Because MR is more immersive than AR. in MR, virtual objects can interact with real-world objects, it doesn’t blocks your view (like in VR), and you don’t have to look up (like in Google Glass) in order to see AR/MR content on top of your real-world view.
And now with the Magic Leap hype, looks like it’s not that revolutionary compared to HoloLens, but I do like what they are doing for creators, and I do enjoy the competition — Hopefully it will help the AR/MR/XR tech to go faster, and evolve exponentially.
What About Apple?
Well, I don’t know if Apple is working on AR/MR Glasses, but if they are, then they’ll have a huge advantage over the competitors, thanks to their ARKit. Developers are already creating so many useful, productive, and innovative AR apps for iOS and Apple’s eco-system. And I believe that Apple will make it easier for Devs to transform their current AR apps from iOS to Apple’s future AR/MR glasses (if they are developing it).
And judging by their career website, looks like they are recruiting relevant talents (such as, Optical Engineers). Btw not only at the Cupertino HQ, but also at the Israeli R&D center in Herzliya (near Microsoft R&D offices). And just like Microsoft Israel, I won’t be surprised if Apple is developing it in Israel, because Microsoft HoloLens was developed in Israel. As you know, Apple has acquired few Israeli Startups in the recent years (The Startup Nation), including PrimeSense, Anobit, etc. So who knows 😉 …?
Prepare for the future, now!
For years, I had so many ideas for AR/MR apps, but now is the time! It's not hard and challenging as it used to be in the past, thanks to AR/MR guidelines, tutorials, and testimonials, you can dive in easily and start transforming your ideas into AR/MR apps. And when you’ve got a good use-case, and an useful idea, it’s time to write them down and pick few good ones before prototyping. So, in the past few months, I’ve been working on few prototypes for AR & MR apps, particularly productive apps that can enhance our capabilities and experiences.
Perhaps, even upgrade some of our human capabilities when it comes to knowledge, skills, Hybrid-thinking. After all, Elon Musk’s Neurolink is just over the corner, and I believe that combining MR with AI will upgrade our capabilities (just like Cyborgs).
I made few prototypes (there are more to come) :
Some are in development, so I’m keeping them to myself (for now 😉).
1. Study with AR/MR glasses
This AR/MR App prototype for students is how I believe we should read books. And also study, and save information in a productive way that will allow us to access that information whenever we need it, by using keywords and Hash-tags. In the future we will have AR/MR glasses (or AR contact lenses) and it will be only natural to access information and knowledge, without having to remember where you’ve read about it, or even without Googling it. This will change the education system, and how we remember and access the things we’ve learned / read. Just like having super powers…
2. Sketch with Mixed Reality glasses
This is how creative people can be productive with AR/MR glasses (or AR contact lenses). Sketch on paper, save it to the cloud, or locally to your glasses. Duplicate, paste, scale, transform, generate, and do things faster without distinguishing between the real-world elements and the virtual world elements. This will change the way creative people do their work. And for the UX designers out there, imagine how cool it will be to get Axure / Sketch features in the real world…. Be creative and productive, Just like a wizard.
3. Control IoT devices with a AR/MR Glasses
This is how I believe we should control our Smart-Home IoT Devices. Not with remote controls and Smartphones, but with your hands, just like a Jedi! In the future we will have AR/MR glasses (or AR contact lenses) and it will be only natural to control everything just like having telekinesis capabilities!
4. Control your Tesla Model S with your AR/MR Glasses
Another quick concept: Control your IoT devices like a Jedi! This time Tesla’s Smart key and IoT connectivity with AR/MR glasses. Now you can remotely watch your internal dash-cam , just like you are sitting in the car. And with Tesla’s SUMMON feature, one day you’ll be able to drive your car remotely!
5. Re-live your memories and moments
Re-live and experience photo albums in AR/MR: Find out location related photos from your friends and family members.
By the way, You are more than welcome to share your ideas, concepts, and Apps you’ve designed / developed for AR, MR, and Spatial Computing…
Let’s see if we can move things faster, and maybe Spatial Computing will become our reality even sooner than we think.
How to start with AR/MR?
- Microsoft HoloLens documentation for Designers and Developers.
- Magic Leap documentation for Designers and Developers.
- Apple ARKit documentation for Designers and Developers.
- Google ARCore documentation for Designers and Developers.
- And read my TL;DR article about AR/MR/VR…
Btw, this entire article was written from my Galaxy Note 9, edited with MS Office for Android, and then published from my Galaxy DeX Station… 😉
If you enjoyed this, read my other articles:
Your Smartphone vs. Your PC
Apple, it’s time to merge iOS with macOS
The differences between VR, AR & MR
Apple is going to reinvent the modular computer
Cross-Platform UX Guidelines (for Devs & Designers)Btw
UX Vs. Cyber-security — Where we draw the line?
Design Thinking as a framework for Innovation
Lifehacks for Productive reading!