Fun with WebAR

Web AR is a type of augmented reality where a user can navigate to a website (in this context, it’s often called a web app) on their mobile phone browser, and use their phone to interact with a physical item in the user’s surrounding environment through augmented reality.

Why are we interested in it?

AR is a hot topic right now in many industries — from tech to media to the medical field, everyone is trying to find a way to integrate AR features. According to a recent CNBC article, Google executive Amit Singh expects that “hundreds of millions” of Android devices will have augmented reality (AR) capabilities by the end of next year. He also said that he expects AR to become a “daily habit” and out of that, monetization opportunities will arise.

Currently, AR experiences are mainly housed in mobile apps. Downloading an app is a high barrier of entry. As an emerging platforms team at a media company, we’re interested in new ways for storytelling that reaches people on the platforms that they’re on with technologies that interest them. To create an app and maintain it for Android and iOS to be used in one longform story would not make sense.

What we set out to prototype

When we decided to work with AR, we immediately knew that a web AR experience would be most applicable for our uses. After some research, we decided to do 2 small experiments. Note: we did not attempt to integrate any animated 3D images in this experiment.

Experiment 1: AR marker with a wide border

• Uses AR.js to create a web app that uses your mobile browser and camera access to scan the AR marker
• We’d create 2 AR markers (created by our amazing designer Adam Baumgartner) with wide borders and use it to trigger an interaction

Experiment 2: AR marker without a border

• Uses Awe.js to create a web app that uses your mobile browser and camera access to scan the borderless AR marker
• We’d use an existing illustration from an ad campaign to trigger the interaction https://www.thrillist.com/viva-los-muertos

What we created

Marker Page (open link on your computer with any browser)

https://s3.amazonaws.com/static-pages.groupninemedia.com/ep-web-ar/public/marker.html

Camera Page (open with either iOS Safari 11+, Android Browser 56+, or Chrome for Android 61+)

http://s3.amazonaws.com/static-pages.groupninemedia.com/ep-web-ar/public/index.html

Instructions:

1. Open marker page on your computer
2. Open camera page on your phone with one of the browsers listed above. If you are using an iPhone, make sure that in your Safari preferences you allow camera & microphone access.
3. You should see a camera on your phone’s web browser after the page opens.
4. Scan the AR marker on your computer with the camera page on your phone. You should now see an astronaut.

Results and what we learned

As you can see, we only completed one demo. So, what happened? Well, let me start with our successful project first.

AR.js worked. It was efficient and fast (~60 fps) and we loved that it was open-source and easily-accessible. With this project, we were able to play around with the ratio of the AR marker size to the size of the 3D rendering. We found that when the 3D rendering got too big for the size of a user’s phone display, the image became unpredictable and glitchy.

Awe.js did not work. The library is outdated and the company is now making a paid GUI tool to create AR interactions with borderless AR markers. Their tool seems great and it’s totally understandable that they’ve stopped updating their public github repo. As a way to try to salvage the 2nd prototype we also tried a few other solutions — including JSARToolkit with Natural Feature Tracking. Unfortunately, the app frequently crashed and we were not able to create a borderless prototype.

Conclusion

Web AR can not yet promise a consistent, production-quality experience that will embed naturally into Group Nine products. While AR interactions built with AR.js and using bordered AR markers create an enjoyable experience for the user, it is questionable whether the experience is polished enough to present to a wider audience.

The AR Marker must be surrounded by a large black border and placed upon a light background, requiring designs to cater to strict requirements in order to make AR integration feasible. In addition, only the most modern browsers support the Web AR technology requirements, and even within these advanced browsers, the AR app quickly drains the device battery. For those who are fully aware of the limitations and potential bugs within the technology, but are still eager to incorporate the technology in a limited fashion, we recommend an integration incorporating bordered marker tracking.