I Made A Thing!

Facial Recognition Edition II

Jack Pincus
Jul 30, 2017 · 6 min read

With an added bonus thing I made, at the end!

Do you like this project? Please share, comment, and recommend!

TL;DR

Use your beautiful face to open doors! Literally. Or keep certain faces on the other side of that locked door. My app makes it super easy to control who is and isn’t allowed to access real life things using facial recognition. You can play with it live here.

What’s the big idea?

I want to be able to confirm identity in places where a traditional keyboard, mouse, and display is impractical, like out in the real world away from a desk.

This is a spin on last week’s ‘I Made a Thing’ post. Reading this will give you a little more context. Starting from where I left off with basic web login using facial recognition, I decided to develop in the direction of making an authentication platform. The premise is that a device can use a photo of a user to determine if they are authorized.

It might be easier to explain this with an example: A door to be specific.

Doors are, at their root, an access control device. You need a key or some other permission - like somebody buzzing you in - to get access to the other side of the door. With the app I’ve designed, a door equipped with a camera can send my app a photo, and the app will tell the door if it recognizes the person in the photo, and whether that person is allowed in.

The web interface for my app allows whoever owns the door to set which specific people can open it. So you could set it to allow a house-sitter/dog-walker/cable-guy in while you are away from the house, and then disable or enable that on a person by person basis. No more lending keys out, or changing out a passcode and sending it to everybody once a month. Each person’s facial features are a unique key which can be enabled and disabled by each device’s owner.

Of course, a door is just one example. Anything with a camera that can connect to the internet can use my web app.

Besides the actual web app, I’ve also built a basic access control device. It allows anything that can be operated electrically to be controlled by the identification service. More on that in the technical section.

In action

So there’s a couple of cool new features.

  • You can register apps or devices that can then call my api to authenticate using a photo.
Decide on a name, and decide on the rejection confidence that the AI should use (basically, how sure do you want to be that you have the right user).
  • You can add or remove users to your app/device that you want to be able to authenticate. My app will send back an authentication confirmation with the user’s first name and email address when it is sent a photo of a user that it recognizes. By default, the owner of the app or device is added as the first authorized user.
There’s a grammatical error in the captions of this gif that is driving me nuts…
  • Users can see a record of when they have successfully been identified by devices. It shows the time, the photo that was taken, and the name of the app.
What my authentication log looks like.

Technical

On Github

A bunch of credit is due to Kairos and their facial recognition API. They are the ones doing the heavy lifting when it comes to actually processing the images. Kairos also has a great pricing structure for hobby projects: personal projects are free with some very reasonable rate limits.

It is also worth mentioning that in limited experimentation I have yet to be able to fool the Kairos API. I already unsuccessfully tried my driver’s license, and a phone screen with my photo to try and spoof my way in last week. More impressively I recently tried holding a high quality photo of myself (albeit several years old, but unmistakably me) up to the webcam, which was also rejected. I don’t doubt that it can be fooled, but kudos to Kairos for having a very robust algorithm.

Rails: from the Rails perspective this modification of the original app from last week is relatively simple. I added models for apps/devices. I then connected Users to apps with three junction tables. One that records the owners of the app/device (Unfulfilled potential. Right now there is no way to add a second owner to the app/device from the web, but the capability exists), one that records the authorized users on an app/device, and the last one that records successful authorizations of the user. This is what the schema looks like:

Created using this awesome site. Worth noting: that site can take a visual representation of your schema, and then WRITE THE SQL TO SET UP THAT DATABASE FOR YOU.

Other challenges: What happens when things don’t work? As apps like this build in complexity, and especially when dealing with an external API, you have to imagine all of the ways that things can go wrong, and then test for them. As mentioned before, setting up a test suite for something that takes a picture through the webcam is REALLY hard. Most of my testing for the webcam features has been manual. As this project gets bigger, finding where in the stack things went wrong gets more and more challenging. This is where it really pays to do things the right way. Following good OO principles and design patterns makes it a lot easier to isolate issues. Lots of credit to the instructors at Turing School who are absolute sticklers for high quality code (much to my benefit, and sometimes to my annoyance).

One more thing: This app would be VERY boring if I hadn’t made any app/device that consumed the API. So that’s what I did. Using a Raspberry Pi Zero W, a proximity sensor, a camera module, and a relay, I can now do some really cool stuff with my own app. I’ll talk more about it in a future post (probably next week). What it does:

  • Senses when a person is within 90 cm. (For fellow Americans: That’s half an inch short of a yard). This is adjustable from 2 cm.
  • Takes a photo when a person is in range.
  • Sends the photo to the web app.
  • Turns on an electrical relay if the person is recognized and authorized to use the device. This relay can be hooked into anything that operates electrically, in this case I’m just wiring it into an outlet. So whatever is plugged into that outlet is only powered when a user is recognized.
  • When the user wants to power down whatever is plugged in, there is a button they push that deactivates the circuit and has the device start looking for people it recognizes again. It can also be set up to timeout after a given time, or to shut down the circuit when the person moves out of range.

Wrap-up

I’m pretty proud of this app. The idea was hatched when I was having a few beers with a group of developers at their office. This was in Vancouver, where beer is expensive enough that the company asks employees to chip in to help cover costs. The developers working there were lamenting not so much that they had to chip in every time they grabbed a drink, but that it was a pain to carry around cash. There was some speculation about using their RFID cards to maintain a ledger. This is what lit the spark that led to this personal project. Find a way to make it easy and painless to identify and authorize people.

Call to action

I love developing and writing about what I’m doing.

  • If you have feedback I would love to hear it, that’s what the comment section is for. Or send me an email: JWPincus@gmail.com
  • If you liked this project, please click that little green heart right below this.
  • If you want to share it with the world: please do. Twitter, Linkedin, Facebook, Myspace, Friendster, Reddit are all options. Not Yo, however.
  • I’m moving up to Seattle, and looking for a new position. I’d love to talk development over coffee, if you’re in the area (even if you aren’t looking to hire). If you’re not near Seattle, I carry a phone with me at -literally- all times. Get in touch!

Jack Pincus

Written by

https://github.com/jwpincus || twitter: @jwpincus || Software Developer

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade