ARCore tutorial

Try-On makeup with Augmented Faces

Kristina Simakova
Jul 6 · 2 min read

Youtube has recently released a new AR beauty try-on feature that lets users try on makeup while watching a video tutorial. In this tutorial, you will learn how to create a similar experience using ARCore and Augmented Faces.

Photo by Gustavo Spindula on Unsplash

Augmented Faces is a subsystem of ARCore and lets your app identify different areas of a face and overlay those areas with textures and 3D models.

Prepare texture material

To be able to build a lipstick try on app a texture is required. We will use a UV texture from a reference face model canonical_face_mesh.fbx. You should get a texture like on the image below. Based on this texture, you can create any overlays you wish.

Reference face 3D model & UV map texture
UV texture as a face texture & Texture example from ARCore sample code

Adding functionality to Android app

  • Dependencies:

implementation "com.google.ar.sceneform.ux:sceneform-ux:1.10.0"

  • Add Camera permission and AR feature to AndroidManifest.xml
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera.ar" android:required="true"/>
  • Add meta-data under application:
<meta-data android:name="com.google.ar.core" android:value="required" />
  • Configure AR session and set it to MESH3D
  • Init a texture for a face filter & add your texture to the drawable folder:
  • Add texture to a new detected face:

Conclusion

Based on the Augmented Faces sample from ARCore, it seems pretty straight forward to track and add any texture to any specific face areas using a UV mapping texture from canonical_face_mesh.fbx. You can find a screenshot from a demo “try-on lipstick” app:

Demo case for lipstick makeup try-on app

Source code:

Kristina Simakova

Written by

Traveler and a big fan of #AR & #ML. Android & Maps Google Developer Expert. Founder at @Reality_Lab_AS. One of @GDGOslo organizers

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade