Vikas Yadav
Vikas Yadav
Published in
6 min readMar 26, 2019

--

🕵️‍♂️ About

ApproachableAR is an online platform for people to learn more about the first wave of mobile AR technologies on Android and iOS. This platform focusses to introduce some technical concepts in simple vocabulary and introduce pragmatic tools to facilitate making.

Duration: 12 weeks

Tool used:

Approachable AR is an outcome of my thesis at CMU School of Design. My objective was to push myself to learn in-depth and furnish a consumable form of the knowledge I obtained. Mobile AR is exciting, maybe not killer right now but I see tremendous potential soon. I chose to learn more about this segment of upcoming mobile technologies with the release of ARKit and ARCore in 2017.

🥅 Goals

Why designers? Because I most identify with the community. Learning is empowering, having in-depth knowledge, invigorates curiosity and play. I have come across many outlets with information on ARKit and ARCore, but none do a good enough job to communicate in most accessible(simple) terms.

🧐 What did I learn?

My first exposure to ARKit and ARCore was through Apple’s and Google’s respective developer conferences. Those sessions were critical to allow me to form a perspective and a game plan for research.

Literature Research

Developer conferences and keynotes are fantastic resources to familiarize with upcoming technologies.

Augmented Reality and Virtual Reality are very different, but similar in ways they allow users to interact and consume. Alex Kipman has been one of the most prominent figures in the development of Microsoft Hololens.

Mike Alger is a designer by training, his VR manifesto is refreshing and throws light on affordances involved with Virtual Reality. He does a fantastic job in manifesting his knowledge about VR into actionable guidelines for people designing for VR.

Martin Schubert over at Leap Motion presented his impressive explorations around hand gestures and interactions during Augmented World Expo.

Studying these people gave me insight into how can I go about understanding MobileAR besides the public documentation provided by Google and Apple. Parsing through product documentation of Google Cardboard, Daydream, and Microsoft Hololens gave me a good depth of tools and workflows involved in their development, such provided tremendous help troubleshooting during prototyping.

Expert interviews

I’m excited about mobileAR and such is a construct of my understanding of the technology. I connected with people working in the industry and academia to get a broader, more holistic perspective. Some things I learned were:

“ Hardware technology will push the canvas for AR and VR more than ever…”

“(about mobileAR) It will just continue to grow from here onwards as the smartphone is ubiquitous and continues to be essential…”

“ as design gets more integrated with the engineering industry, we need better tools to allow designers to make and explore this medium…”

Technical Research

Developer documentation does point out the limitation of mobileAR technology; however, the extent of those limitations are still unknown. To better understand the restrictions, I conducted some technical experiments to best surface quantifiable limits.

This blog contains my experiments around Scaling(how does mobileAR scale content concerning camera feed), environmental conditions(lighting and handling angles) and best practices(a hybrid of recommendations from documentation plus my inferences from experiments).

Checkout technical research blog here

👏 Key insights

Research, by far, informed three points:

1. There are numerous bits and pieces one needs to know before playing with mobileAR. Not knowing can leave you confused and bewildered

2. Most avenues use technical jargon to explain concepts, can be made simpler and more digestible

3. Making/prototyping can infuse tonnes of confidence in people trying to play with mobileAR

Core principles for Approachable AR

These core principles helped me think about how to package my thesis to make it most useful to the design community.

👨‍💻 Prototyping

The following are some prototypes I built to understand the pipeline. Ranging from standalone apps to tweaking the template ARKit app, it was valuable hands-on training I needed.

Some working app prototypes on iOS

At a certain point, I started putting together tutorials; however such was a time-intensive process, especially while trying to graduate and finishing a thesis. I put together one tutorial which can allow anyone to pop in custom content within ARKit template app, can be found here

My typical prototyping workflow cascaded from designing on Sketch, referring components/code bits from frameworks and developing in Xcode/Unity(with ARKit plugin).

📦 Packaging

The biggest question 4 weeks before the deadline was, “How do I want to package my knowledge about mobileAR?”. I wanted a scalable medium that can reach a wider audience. Hence, a website!

Some early concept sketches for website layout

There is much to talk about design and organization of content on the website, but such is a super long conversation, let’s do it in person 😉

👀 User Testing

I was able to put together a site in 2 weeks with two more weeks in hand to get some feedback. This was the magical phase where resultant tweaks made a world of difference to the usability of the website. Some input is as follows:

Based on the feedback, I designed a new onboarding experience and tweaked static illustrations into animated gif’s one week before the final showcase.

A new on-boarding experience

The new on-boarding aimed at establishing correct expectations from navigational menus on either side of the page

Animated illustrations

World Tracking [Before / After]
Scene Understanding [Before / After]
Plane Detection [Before / After]
Hit Testing [Before / After]

🤩 Impact

The designers loved it! The new onboarding was effective in conveying navigation across the site while animations brought more life to the technology section. There is more work to be done to expand the making aspect for ApproachableAR. Please get in touch if you have any ideas or want to contribute 🙏

--

--