Build a face-detecting React Native selfie cam from scratch in minutes with Exponent

TLDR: Install exponent, initialize an app, and replace the contents of App.js with this gist.


React Native Face Detecting Auto Selfie Cam

I recently had a request to build an app that, among other things, would take a photo only if it recognized a face in the frame and was relatively stable. In this article I’m going how to show you how to do just that in a remarkably short amount of time.

As background, I’ve been away from React Native for more than a year — having to some extent sworn off projects with lots of boot-strapping, dependencies, build system requirements, and other things that React and native development are known for. But I’ve also been very immersed in Machine Learning and Computer Vision, and so I headed back into the fray, fully expecting a lot of headaches getting up and running. They never came.

I didn’t want to deal with a lot of the hassles of setting up XCode, so I decided to look at Exponent. It gets around a lot of the hassles of provisioning and sharing and testing prototypes. But my app needed to use Computer Vision, Machine Learning, and native Camera APIs, which I thought would be a lot to ask of Expo. Turns out, that stuff is all available out of the box. So here’s how it’s done.

Installation

Download the expo CLI: npm install -g expo-cli

Within your desired folder path in the command prompt, type expo init . Give your project a name (e.g. CameraExample)and let the installation complete.

cd into the project directory and run expo start . This will start a local web server as well as a tunnel that is publicly accessible from other devices. Out of the box you’ve got react, react native, babel, webpack, live-reloading, a local web server, and no need to provision or set up XCode. That alone is worth days of development time!

Now for the code.

Code

Everything in Exponent starts with App.js . Open up that file in your code editor.

Let’s start with dependencies.

Dependencies

import React from 'react';
import { StyleSheet, Text, View } from 'react-native';
import { Camera, Permissions, FaceDetector, DangerZone } from 'expo';

We need react for react native, and we’ll need some basic styling, text, and container elements for a simple UI.

Then, from expo, we’ll need access to the camera, the face detection algorithms (provided by Google Moble Vision), and I’m using DangerZone to get access to Device Motion, because I want to ensure that the camera is stable as well before taking a photo.

Props and State

We’ll set up a basic class structure with props and state that we’ll need to keep track of:

export default class CameraExample extends React.Component {
static defaultProps = {
countDownSeconds: 5,
motionInterval: 500, //ms between each device motion reading
motionTolerance: 1, //allowed variance in acceleration
cameraType: Camera.Constants.Type.front,
}
state = {
hasCameraPermission: null,
faceDetecting: false, //when true, we look for faces
faceDetected: false, //when true, we've found a face
countDownSeconds: 5, //current available seconds before photo is taken
countDownStarted: false, //starts when face detected
pictureTaken: false, //true when photo has been taken
detectMotion: false, //when true we attempt to determine if device is still
motion: null, //captures the reading of the device motion
};
countDownTimer = null;
}

Among our props, I’ve chosen to have a countdown. Once it detects a face, it gives the person 5 seconds to get pretty. You could optionally just have it take the photo.

I’m also capturing the device motion (which you could skip as well) but I only want it to update every half a second. My app actually involves some physical hardware that allows a person to put their phone in a dock, thus enabling it to be absolutely still before the photo is taken, so frequent calls to device motion are not necessary. I also have a motionTolerance which I’ve set at 1, meaning if the device moves no more than 1 degree in any axis between updates, we’ll consider it still.

Lastly, we want a front vs rear-facing camera so that people can see themselves in the selfie.

As for state, there’s a lot we need to keep track of: whether we have permission to use the camera, whether we are currently detecting faces, whether we’ve found a face, the current countdown and whether it has started, whether we’ve taken a photo, whether we’re detecting motion (no longer necessary once we’ve taken a photo), and the state of the device motion reading.

countDownTimer = null; just keeps a placeholder for our countdown interval so that we can easily cancel it or restart it if needed.

Lifecycle Hooks

React has some great Lifecycle Hooks we’ll want to tap into to get everything started:

async componentWillMount() {
const { status } = await Permissions.askAsync(Permissions.CAMERA);
this.setState({ hasCameraPermission: status === 'granted' });
}
componentDidMount(){
this.motionListener = DangerZone.DeviceMotion.addListener(this.onDeviceMotion);
setTimeout(()=>{ //MH - temp - wait 5 seconds for now before detecting motion
this.detectMotion(true);
},5000);
}
componentWillUpdate(nextProps, nextState) {
if (this.state.detectMotion && nextState.motion && this.state.motion){
if (
Math.abs(nextState.motion.x - this.state.motion.x) < this.props.motionTolerance
&& Math.abs(nextState.motion.y - this.state.motion.y) < this.props.motionTolerance
&& Math.abs(nextState.motion.z - this.state.motion.z) < this.props.motionTolerance
){
//still
this.detectFaces(true);
this.detectMotion(false);
} 
}
}

As soon as we know our app will mount, we’ll want access to the camera. We need permissions for that, so we’ll initialize a prompt on startup. Once the user agrees, we can use their camera.

Once we mount, we initialize the device motion listener. I’ve got a timeout of 5 seconds before anything happens, which is kind of a hack. Ideally we’d just want some sort of button or prompt to start the experience. I just didn’t want it immediately counting down before someone had time to realize the app had even loaded. When we do have device motion, we just want to save that to the state object.

onDeviceMotion = (rotation)=>{
this.setState({
motion: rotation.accelerationIncludingGravity
});
}

Then, in order to compare the current state of device motion vs the last state, we use the componentWillUpdate lifecycle hook. This gives us access to nextProps and nextState before they are updated, so that we know what is about to replace this.props and this.state . Here, I’m making sure that we want to be detecting motion (this.state.detectMotion ), making sure our motion states exist, and saying that if the difference between this reading and the last is no greater than our motion tolerance in any axis, then we can begin detecting faces, and stop detecting motion. Those two functions look like this:

detectMotion =(doDetect)=> {
this.setState({
detectMotion: doDetect,
});
if (doDetect){
DangerZone.DeviceMotion.setUpdateInterval(this.props.motionInterval);
} else if (!doDetect && this.state.faceDetecting) {
this.motionListener.remove();
}
}
//DETECT FACES
detectFaces(doDetect){
this.setState({
faceDetecting: doDetect,
});
}

Face Detection

Our face detection all happens via the Camera component:

<Camera
style={{ flex: 1 }}
type={this.props.cameraType}
onFacesDetected={this.state.faceDetecting ? this.handleFacesDetected : undefined }
onFaceDetectionError={this.handleFaceDetectionError}
faceDetectorSettings={{
mode: FaceDetector.Constants.Mode.fast,
detectLandmarks: FaceDetector.Constants.Mode.none,
runClassifications: FaceDetector.Constants.Mode.none,
}}
ref={ref => {
this.camera = ref;
}}
>

We set its type to front facing camera, give it a callback when it recognizes a face, set its mode to fast (you could use accurate if you’d rather), and turn recognition of individual face landmarks off (eyes, ears, nose, etc) — we’re not interested in what’s happening on a person’s face, just that there is one in frame. runClassifications does things like see if eyes are open, if a person is smiling, etc. I don’t need those for mine, so I turn them off. You can see documentation for faceDetection here.

When we do detect a face, I have this callback:

handleFacesDetected = ({ faces }) => {
if (faces.length === 1){
this.setState({
faceDetected: true,
});
if (!this.state.faceDetected && !this.state.countDownStarted){
this.initCountDown();
}
} else {
this.setState({faceDetected: false });
this.cancelCountDown();
}
}

The face detection returns an array of faces, with a whole bunch of information about them (if we hadn’t disabled it all). All my app cares about is that there is one (and only one) photo in the frame. If you wanted to allow more you would change if (faces.length === 1){ to if (faces.length >= 1){ .

If there is a face, we start a countdown to let the person know we’re about to take their picture. Otherwise, we cancel any pre-existing countdown, and restart it when we detect a face again.

Handling Countdown

initCountDown = ()=>{
this.setState({
countDownStarted: true,
});
this.countDownTimer = setInterval(this.handleCountDownTime, 1000);
}
cancelCountDown = ()=>{
clearInterval(this.countDownTimer);
this.setState({
countDownSeconds: this.props.countDownSeconds,
countDownStarted: false,
});
}
handleCountDownTime = ()=>{
if (this.state.countDownSeconds > 0){
let newSeconds = this.state.countDownSeconds-1;
this.setState({
countDownSeconds: newSeconds,
});
} else {
this.cancelCountDown();
this.takePicture();
}
}

Nothing fancy here (in fact we could make it much more elegant). Just setting an interval to mark down the seconds, and canceling that interval if our faceDetected state has changed to false. When our countDownSeconds reaches zero, we stop the countdown interval from happening further, and take the picture.

Taking the picture

takePicture = ()=>{
this.setState({
pictureTaken: true,
});
if (this.camera) {
console.log('take picture');
this.camera.takePictureAsync({ onPictureSaved: this.onPictureSaved });
}
}
onPictureSaved = ()=>{
this.detectFaces(false);
}

My app is not allowing for any repeat pictures to be taken, but ideally you’d want everything to be reset again and provide some sort of way to take another picture.

We call the takePictureAsync function from exponent, which you can read about here. It saves a photo to the app’s cache, and gives us a callback when that picture is saved. You’ll notice in my onPictureSaved there are no parameters, but that callback gives us data about the picture, so if your app needed to do something with that picture data, you’d need to put all that logic here, along with some sort of pictureData parameter. For simplicity, my app just stops face detection here.

That’s all there is too it! Pretty advanced stuff for just a couple hundred lines of code, made very easy thanks to Exponent (and very stateful, thanks to React).

Here’s the whole code: