Resn’s Little HelpAR is an update of a very Resn yuletide tradition. This year in trembling anticipation for AR on the Web we forged a new experience that was built on the last, one which tips a bell-festooned hat to the emerging world of augmented browser experiences.
AR Research
First things first, we asked ourselves what Augmented Reality actually means and how it works.
Simply put, AR can project digital environments and information over the real, physical world. Currently it is most common to use a smartphone to use this technology. Sure, glasses are out there but no one owns a pair.
The two current major AR platforms, ARKit and ARCore, are already quite well documented. These documentations and other information sources confirm that augmented reality is most often based on the Simultaneous Localisation And Mapping (SLAM) technique. The core principle of SLAM is to align the digital world with the physical one. ARKit, for example, achieves this by combining movement estimations (from accelerometers) and feature point tracking through computer vision. This specific combination of input and data processing is called Visual Inertial Odometry.
FAR (Faux Augmented Reality)
Resn is an internet company — We love the Web for its versatility and accessibility. So it follows that if we were to make an AR experience then it would need to work in browser.
We explored the possibilities of the WebXR spec in experimental browsers, but wanted the Little HelpAR to be accessible for a broad, festive audience. In response to that internal brief, we started researching the possibilities and restrictions of what we coined: Faux Augmented Reality (FAR).
After prototyping multiple odometry and tracking systems we learned that we wouldn’t have the time (Christmas was fast approaching!) and capacity to implement any proper position tracking system in a mobile browser. Luckily with the gyroscope we still had access to accurate orientation data; that’s 3 out of the 6 degrees needed for a full AR experience, the other 3 we figured we’d handle with some good ol’ fashioned creativity.
FAR creative solutions
We realised that when we placed objects in a FAR scene they would not stick to any position, or scale to fit the ‘real’ world.
In AR, casting shadows underneath models boosts the illusion of alignment and place. However in our 3-degrees-of-freedom FAR experience shadows did the opposite, since it’s impossible to anchor a model to a specific real-world position.
But Jethro is a magical elf with magical powers and we got around this issue by making him levitate in space. With this approach we didn’t have to anchor Jethro to any surface and the resulting illusion of him being in the real world is much more convincing.
Jethro’s integration felt very believable… up until the point that we would try to walk towards him at which point he kept the same distance at all times.
To counter this we used the accelerometer data to update Jethro’s position and apply forces to his limbs and head, giving the illusion of ‘swimming’ away from, or towards annoying human beings.
Content and interactions
Little HelpAR was a sequel of sorts to last years wild experience. With that in mind it was necessary for us to bring back the same visual style and absurd elements used to create the myth of Jethro the elf with the massive addition of another dimension, taking things up from 2D to 3D.
We used Mixamo to automate the skeleton construction with all the textures painted by hand. His shiny new Gnome-Cler jacket for instance, had a matcap material applied to give it that quintessential reflective property of certain other high profile brands.
To separate the ABOUT and SHARE sections, but still integrate them in the FAR experience we created a different world for each which could be entered by summoning a portal and entering therein…
Inside, users can browse a gallery simply by looking around with their video feed. To give the audience a helping hand we made the interactive elements snap to the camera’s rotation when a set distance threshold is passed. This gives a pleasing interaction and makes it easier to focus on a particular element; we coined this the ‘sloppy lock’.
Conclusion
We had a lot of fun making this little project and we learned a lot about AR and its current possibilities and restrictions on the web. We also learned how we can create accessible and creative FAR experiences in mobile browsers for the ‘right now’ educating and preparing and eager audience hungry for fun augmented experiences grounded in creative and story.
We are extremely excited about Augmented Reality and we can’t wait for full WebXR integration in all major browsers. In the meantime, Resn’s Little HelpAR is our Christmas cARd to the world.
