Creating A Thing That Looks At You

Tom Jacobs
Jul 27, 2017 · 4 min read

This series of posts chronicles my path to Create A Thing That Looks At You.

My eye

Using my freshly printed gimbal mount, a Raspberry Pi, a Pi Cam, Dlib’s Face Recognition library, two gimbals, two gimbal controllers, and a PWM generator board, the plan is to make a 3D printed object that detects faces, and points itself directly towards the detected face. If more than one face is detected, it just picks the most active one, that is, the most rapidly changing position one — so it appears to take an interest in interesting things.

We definitely want to minimise the creepy factor here. None of this.

All I see is table tennis paddles

The plan
The plan is to have it work by having Dlib find faces in the camera frame image, calculate the centre x, y co-ordinates of a selected detected face, and subtract the x, y co-ordinates of the centre of the whole camera image. From that we get the x, y amounts to need to move the pan (x) and tilt (y) gimbals to have it place that face in the centre of the camera image. We feed these into a PID (Proportional, Integral, Derivative) controller written in Python, and output these values into the PWM generator board. This board generates PWM signals which feed into the gimbal motor controllers, and these send the power signals along three wires each to the gimbals to silently adjust where the camera is pointing.

Well that’s the plan anyway. Let’s see how far we get.

I see software library installation issues in your future

First step. Get Dlib happening on the Pi 3.

pip install face_recognition

Python: import face_recognition

Did it work first time? Of course not! Let’s go through the errors.

Boost python library not found

No lapack/blas resource found

Looks like we need some more stuff.

sudo apt-get install libboost-all-dev

100+ packages to install, cool.

And we also need some lapack.

sudo apt-get install build-essential cmake libatlas-dev libavcodec-dev libavformat-dev libgtk2.0-dev libjpeg-dev libswscale-dev liblapack-dev

Run it again. Seems that the cpp compiler is running 100% CPU, but it’s not saying it’s compiling anything. Oh well, we’ll let it go and do its thing.

Actually, it wasn’t two hours later. Three hours and two Pi freeze restarts later, we still don’t have Dlib installed.

Ah. It’s too much for the poor Pi to compile Dlib. Got it. Read the instructions for setting up some swap space on the Pi to compile Dlib. The important part is to increase swap space to a gig, from 100 meg.

Update: The above only works on Raspbian. On Ubuntu, we do:

Hooray! Dlib compiles. We move onto hardware again. Next, we plug the Pi cam into the Pi.

And now we write a little Python to detect some faces.

We install this:

sudo pip install picamera[array]

And give it a run.

Bingo!

Found 1 faces in image.
A face is located at pixel location Top: 82, Left: 124, Bottom: 211, Right: 253
Movement: 3.1, -0.9

Great success. And now we take those movement numbers and move our camera.

Australian Robotics Society

The Australian Robotics Society is a community of robot builders.

Tom Jacobs

Written by

Making robots.

Australian Robotics Society

The Australian Robotics Society is a community of robot builders.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade