Resn’s Little HelpAR

Resn
Resn
Dec 21, 2018 · 5 min read

Resn’s Little HelpAR is an update of a very Resn yuletide tradition. This year in trembling anticipation for AR on the Web we forged a new experience that was built on the last, one which tips a bell-festooned hat to the emerging world of augmented browser experiences.

AR Research

Simply put, AR can project digital environments and information over the real, physical world. Currently it is most common to use a smartphone to use this technology. Sure, glasses are out there but no one owns a pair.

The two current major AR platforms, ARKit and ARCore, are already quite well documented. These documentations and other information sources confirm that augmented reality is most often based on the Simultaneous Localisation And Mapping (​SLAM​) technique. The core principle of SLAM is to align the digital world with the physical one. ARKit, for example, achieves this by combining movement estimations (from accelerometers) and feature point tracking through computer vision. This specific combination of input and data processing is called ​Visual Inertial Odometry​.

FAR (Faux Augmented Reality)

We explored the possibilities of the WebXR spec in experimental browsers, but wanted the Little HelpAR to be accessible for a broad, festive audience. In response to that internal brief, we started researching the possibilities and restrictions of what we coined: Faux Augmented Reality (FAR).

After prototyping multiple odometry and tracking systems we learned that we wouldn’t have the time (Christmas was fast approaching!) and capacity to implement any proper position tracking system in a mobile browser. Luckily with the gyroscope we still had access to accurate orientation data; that’s 3 out of the 6 degrees needed for a full AR experience, the other 3 we figured we’d handle with some good ol’ fashioned creativity.

FAR creative solutions

The same model looks big on the floor and small on the desk

In AR, casting shadows underneath models boosts the illusion of alignment and place. However in our 3-degrees-of-freedom FAR experience shadows did the opposite, since it’s impossible to anchor a model to a specific real-world position.

But Jethro is a magical elf with magical powers and we got around this issue by making him levitate in space. With this approach we didn’t have to anchor Jethro to any surface and the resulting illusion of him being in the real world is much more convincing.

Jethro’s integration felt very believable… up until the point that we would try to walk towards him at which point he kept the same distance at all times.

Left: without acceleration physics — Right: with acceleration physics

To counter this we used the accelerometer data to update Jethro’s position and apply forces to his limbs and head, giving the illusion of ‘swimming’ away from, or towards annoying human beings.

Content and interactions

We used ​Mixamo​ to automate the skeleton construction with all the textures painted by hand. His shiny new Gnome-Cler jacket for instance, had a matcap material applied to give it that quintessential reflective property of certain other high profile brands.

To separate the ABOUT and SHARE sections, but still integrate them in the FAR experience we created a different world for each which could be entered by summoning a portal and entering therein…

Inside, users can browse a gallery simply by looking around with their video feed. To give the audience a helping hand we made the interactive elements snap to the camera’s rotation when a set distance threshold is passed. This gives a pleasing interaction and makes it easier to focus on a particular element; we coined this the ‘sloppy lock’.

Conclusion

We are extremely excited about Augmented Reality and we can’t wait for full WebXR integration in all major browsers. In the meantime, Resn’s Little HelpAR is our Christmas cARd to the world.

View on mobile — littlehelpAR.resn.global

Resn

Infecting your screen for your enjoyment.