Create an HD AR facemask for iPhones using Apple Reality Composer (no coding required)
In this tutorial, we’ll create a super High Definition AR facemask using Apple AR technology. This process is actually made simple thanks to the Reality composer.
Reality Composer is an iOS app you can use to create diverse AR experiences on your iPad. It can also be used on iPhones !
What we’ll be using
- Reality Composer
- Download the app following this link : https://apps.apple.com/us/app/reality-composer/id1462358802
- A LiDAR capable Apple device to watch
- An iPad Pro. (You can also use any iPhone 11 and and newer models)
- A note on LiDAR : LiDAR or True Depth Camera/Depth Sensing cameras are in general not needed if you want to make a Face Anchor AR experience on other frameworks like Lens Studio or Spark AR.
- ‘Fun’ Fact : Even though a Face Anchor AR experience created in Reality Composer requires a device with LiDAR to activate on your face, by default the experience will automatically switch to your back camera (which does NOT have the LiDAR capability), placing the mask as a world object…
Marketing world be like ;)
The Objective
We want to get a model we’ve created in Blender (shown below) into Reality Composer, keeping as much quality as possible.
Alrighty, enough setup for now, let’s dive in!
Part I : Blender
1) Export
We’ll start by exporting our 3D model from Blender as a .glb file.
IMPORTANT: To be able to export as .glb, make sure all your files and textures are in the same folder. It won’t work otherwise.
2) Convert your export to .USDZ
Reality Composer only accepts .usdz files, so any 3D model will have to be converted to that format. There are several ways to do this, including the Reality Converter (https://developer.apple.com/news/?id=01132020a).
For this example I used spase.io. In order to convert your file, drag and drop the .glb version of your model here : https://space.io/playground
This will generate a preview of your model.
Click on ‘Make usdz’ then ‘Download .usdz’
NB: I’m working on a windows machine so I downloaded the .usdz file and sent it to my iPad on which Reality Composer is installed.
3) Transfer the converted model to your iPad
Download the .usdz file on your iPad or iPhone.
NB: before opening your file in Reality Composer, you can already preview your model by just clicking on the download file and clicking ‘View File’. This will automatically open your camera with a little ‘AR’ button in the top, at the center of the screen.
Part II: Onto Reality Composer!
Open Reality Composer on your device of choice and click on the ‘+ Create
Document’ icon
Next select ‘Face’
You’ll then end up on a blank scene with a relaxed facemesh as shown just below. Delete anything unnecessary for what we’re trying to achieve from here by clicking on it and choosing ‘Delete’.
Now click on the ‘+’ button on the top right, next to the play button.
Click on the ‘+’ button in this new pop up to import your own custom 3D model (items already imported are also shown here). Find the file on your device and confirm the import.
After the import is done, click on your model to make it show in your scene.
Often the model will be at the wrong scale and/or rotation when imported initially. The positioning, scaling and rotating of the object all work very similarly to other 3D programs.
NB: In case you need help with the scaling, rotating and positioning of your own model, I recommend this tutorial : https://www.youtube.com/watch?v=4A2GlmNNkvs
Part III: Export and trying out your AR experience
Position your Object on the face (or surface) as you like and export it.
To export your project and share it with your friends, click on the three dots in the right corner.
Then click on ‘Export’
A new window will open.
Click on ‘Export’ again and you will get the option to send it to various other apps such as Messenger.
Important note : We started this project in Reality Composer by selecting the ‘Face’ option. The other anchor options are ‘Vertical’, ‘Image’ and ‘Object’. Since we had chosen ‘Face’, the ‘Face’ anchor is selected. You won’t be able to project the mask on your face to test it before exporting it if your device does not have LiDAR. You also won’t see the AR preview button on the top of the screen if your device doesn’t feature LiDAR technology.
If the person receiving the project has a phone with True Depth camera like LiDAR, they will be able to try it on their face directly by clicking on the ‘AR’ button. The camera will automatically switch to the Front Camera and place the effect on the user’s face.
Part IV: Time to brag
When the process is funny and good looking. The sharing process implies to record yourself and the recording will end up in your camera rool for you to use it on any social you like to play with.
This new toy open so many new avenues. Creating unique ar effects for clients that could keep them on their phones, is one but Apple also announced that soon we’ll be able to upload AR experiences made with RealityKit from iPads directly to the app store.
Please tag @popul_ar.experience if you share what you made on socials.
If you have difficulties while building this, don’t hesitate to post a ticket on the Lab, we’ll help you :)
And of course, if you don’t have high res models to test out with, you can find some fabulous ones made available on Sugar by our extra talented Artistic Director: Sarah Mayer.
NB: When exporting the project and sending it to yourself or a friend (the file extension is .reality), it will download it like any other file and save it in the downloads folder.
NB2: If the phone you’re using to try it does not have true depth camera like a LiDAR, the effect can still be displayed as a world object on a surface the device picks up, such as a table or the floor.
Credits
