Facebook’s AR Day

Aahir Giri
Sep 5, 2018 · 4 min read

Facebook is investing significantly in Augmented Reality and their platform, AR Studio — which is the company’s big response to ARKit and ARCore by Apple and Google respectively. With AR Studio having been in the market for roughly a year, Facebook recently held AR Day at Bangalore to share the company’s work and roadmap on AR which I had the chance to attend. Here are a few things that I found interesting.

The event was aimed primarily at developers, who can potentially leverage Facebook’s massive user base around the world by adopting it for their products and also testing and validating the quality of the AR platform that Facebook is continuing to build. There was a fair amount of emphasis on plug-and-play, which I feel makes AR more appealing and less intimidating for developers.

We got to experience the possibilities of the front camera, where face tracking is used to add filter masks. This is something that’s been used widely by the public, from the hay days of Snapchat, years ago. There’s a lot of scope for creativity in these filters, such as painting your face with colours of your favourite sports team, or an elaborate colourful turban that sits on your head.

Great progress has been made in face detection, making the whole experience look more real than ever. In addition to this, they have developed hand movement tracking which when combined with effects can pack a punch. Imagine conjuring flames out of thin air, and holding it.

Next comes world tracking, which is the most widely known use case of AR — visualising an augmented world containing 3D objects around us through our phone lenses. Here, the tool is capable of detecting human bodies in addition to surface detection. This opens up the possibility of segmentation, which means changing the background to a rock show stage setting while you’re standing in your room and singing into a mic, for example.

The third, and what I find the most exciting feature, something Facebook terms evolutionary, is target tracking. Imagine scanning a car on the road, and many details immediately popping up around the car on your screen, starting from technical specifications to price and mileage.

One of the presentations involved a Hershey’s syrup bottle. As soon as your phone detects the bottle, a recipe video starts playing immediately inside your screen next to the bottle in thin air.

Now, how exactly can Facebook AR be used for businesses?

The answer is marketing, via Facebook’s suite of apps thereby reaching a potential 1.5 Bn users. Ads on your timeline, which typically have a static image and text with a call-to-action button, can now be an AR experience. If you’re used to furniture ads on your Facebook feed, you could be seeing interactive advertisements of how a particular sofa would look in your living room, and actually trying it out before you decide to buy.

Messenger bots are alternate means to market and sell products, as shown by Nike’s successful shoe drop campaign.

Deep links can be placed in both apps and websites, which will open the Facebook camera internally, fully loaded with everything the developers have built to showcase their product in AR.

The whole idea is to change the way people interact with their phones — from tapping, scrolling and sliding to now moving your arms and entire bodies. And Facebook seems to have made a powerful, user friendly interface involving a great deal of drag and drop functions along with JavaScript that can be deployed on any device irrespective of the computation power, and rendered on the Facebook Camera.

AR Studio supports SketchFab integrations if you’re not sure what you’ve built works right and want to try out some existing 3D models. It also supports file imports in Collada, FBX and GLTF formats. Add to this another piece of big news, for Windows developers: AR Studio will be available on the platform very soon.

This event has served its purpose by getting developers like me excited. Now I’m going to jump right in and explore more. Facebook is betting big on AR, and one way to succeed is to make it more accessible than ever before.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade