Bringing Art to Life with Facebook Camera Effects (Tutorial)
3D filters and effects seem to be all the rage on social media. At first, it looked like Facebook was copying Snapchat with its stories and camera features, but in an interview with Techcrunch, CEO Mark Zuckerberg explains what Facebook is up to: Unlike other apps, they want to establish an open platform on which third-party developers can publish and contribute their work. As Facebook director Ficus Kirkpatrick said, “We want artists, developers, brands and more creators to be able to build and share amazing AR experiences.” Facebook’s logic in opening AR Studio and Frame Studio to all creators and developers is that its feature will find many further applications than if it was created centrally.
In the following article, we share with you our research results on Facebook’s camera effects as well as two tutorials — one for the Frame Studio and one for the AR Studio. Have fun experimenting with the effects and bringing art to life!
Feature
In December 2017, Facebook rolled out World Effects technology in AR Studio. The idea here was to let people augment their surroundings with 3D objects instead of using only their selfies. Herein, the user could add a floating heart or a 3D word bubble like love, bae, miss youto the subject in the picture. And of course, it is widely known that Facebook World Effects is yet another feature copied from Snapchat.
It is an advanced AR Target Tracker configured with face-hand-body tracking and more movements to AR experiences. Likewise, the Patch Editor tool enables creators controlling audio, edit materials, add interactions, or design customized shaders without even writing a single code.
Background Segmentation, introduced in AR Studio enables to (as the name suggests) segregate background and people. The administrators of Facebook Pages publishing AR effects will soon be able to track reach and engagement of their AR creations. Unlike Google’s ARCore and Apple’s ARKit, Facebook efforts are directed towards a different objective, reports Engadget.
Pro’s: to visually brand the client’s Facebook Stories (and enable their fans to do so, too).
Here you can have a look at examples of how brands have used it:
TUTORIAL I — Frame Studio
Turn your art into frames for profile pictures or for people to express themselves within the Facebook camera (NOTE: no moving objects).
1.Draw an artwork (outside of Facebook platform)
Save it as .png or .jpeg .
2. Upload it to the Frame Studio
3. Create frame
4.Save draft
5. Preview in phone
TUTORIAL II — AR Studio
Powerful, fully customisable tools for 3D artists and developers to create masks, animations and advanced AR effects. Downloadable app.
Tutorials: https://developers.facebook.com/docs/ar-studio/tutorials
Inserting a face tracker
To get started, open AR Studio and create a new project. The face tracker is the foundation of the mask. To insert one into your scene:
1. Click Insert.
2. Select Face Tracker.
3. Click Insert.
You should now see axes on the face in the middle of AR Studio. This section is called the Viewport. You can use it to visualize and interact with your scene, together with the simulator.
You’ll also see a face tracker listed in the tab on the left. This is called the Scene tab. It lets you manage the hierarchy of objects in your scene.
Inserting Face Meshes
Next, insert a face mesh. This is a 3D object.
To make the face mesh track your face, you’ll have to insert it as a child of the face tracker. To do this:
1. Select Insert.
2. Select Face Mesh.
You should now see a face mesh in your scene. You can also create relationships between objects in the scene by dragging one on top of the other in the Scene tab.
Importing Texture Files
Textures and materials are inserted from the Assets panel. This panel lets you manage the files you’ve imported, and the materials you’ve created using AR Studio.
Textures can be either JPG or PNG. You’ll need to create textures in advance. For this tutorial, they’ve provided a texture in the sample folder (moustache.png).
To import your file:
1. Go to Assets.
2. Click +.
3. Select Import Assets.
4. Open the PNG from the sample content folder.
You should see moustache.png in the Textures folder.
You can also import assets by dragging and dropping files into the Assets panel from your computer.
Creating and Editing Materials
Now your texture is in AR Studio, you’ll need to create a material to apply it to. To create a material:
1. Go to Assets.
2. Click +.
3. Select Create New Material.
You should see the material you created in the Materials folder. The material’s properties should be displayed in the Inspector panel on the right.
The Inspector panel lists the properties of the selected object, asset or layer and gives you the tools to edit them.
The Shader Type for materials is set to Standard by default. AR Studio includes several different built-in shaders , but the best type of shader to use for basic masks is the face paint material. To change it, click the dropdown menu next to Shader Type and select Face Paint.
Next apply the texture to the material. Click in the gray box next to Texture and select the texture you imported from the menu.
Now you’ve applied your texture to the material, you can apply the material to the face mesh:
1. Go to the Scene tab.
2. Select your face mesh.
3. Go to Material in the Inspector panel.
4. Click + next to Material and select the material you created from the menu.
You should now see your material tracking your face.
Previewing Effects
You can preview what your effects will look like on other people and different mobile devices.
You can also preview your effect on different sample videos with a range of skin tones, facial hair, glasses and hair styles, so you can be confident your effect works well for everybody. Click Video in the Toolbar, and select the one you want to use from the menu.
You can also add your own video. To do this:
1.Click Video.
2.Click +.
3.Choose your video from your computer.
The Aspect Ratio menu in the Toolbar lets you resize both the Viewport outline and the Simulator to match the screen dimensions of common mobile devices. To resize the outline click Aspect Ratio then select the name of the mobile device you want to use.
If you want to preview your effect on an actual mobile device, download and install the AR Studio Player app for iOS or Android. Find out how to use it in our AR Studio Player tutorial.
In sum, we like the idea of crowdsourcing development and find it a totally cool gimmick that you can have some fun with or put to good use at your company.
What do you think of the Frame Studio and the AR Studio? Have you tried it out? We look forward to hearing about your experiences in the comments section below.
