Build a Production Ready Face Detection API
Part 1: Architecture of the API and building the face detection model
With recent advancements in AI, there has been an influx in applications ranging from computer vision, speech recognition, generative adversarial networks. Zooming into computer vision, the has been an explosion of research that have led to possibilities in applications that were only envisioned in sci-fi movies such as driver-less vehicles, advancements in medicine, remote sensing and mass surveillance.
As a software engineer in the process of transitioning to Machine learning engineering, there were a couple of challenges that I face that were trivial but gave me a bit of a challenge. These issues included the fact that, yes I could train a basic model to do basic stuff, but how could I use this model, in a real world application? This gave me the motivation to explore how machine learning models can be used in a production like environment in a scalable and performance conscience manner.
I have deep interest in computer vision and as part of a series on how to tie DevOps to machine learning, I have put up this article as an example on how to create an ML pipeline for facial detection. In this article, we are going to build a simple facial detection API that will be exposed using a REST endpoint that will be used by a simple web/android app (part II) of this post.
Prerequisites
- Python3
- Docker
- Django
The only prerequisites needed are basic knowledge of python and Docker. In-terms of experience, you don`t need to be an ML ninja to implement our facial detection API.
Requirements
In order for us to have a clear picture of what is required, we shall begin by first listing down all the requirements that the API will need to satisfy. below are the requirements:
- The API needs to be accessible via an HTTP REST endpoint
- We need to deploy the application to either of the available cloud providers
- The system needs to have scalability, fault tolerance and availability in-baked.
Face Detection
We are not going to go deep into what facial detection is, but as a simple definition, a facial detection task in an object detection task that identifies faces in an image.
There exists numerous facial detection models, however, for our task, we are going to use the Multi-task Cascaded Convolutional Networks (MTCNN) model (white paper -> https://arxiv.org/ftp/arxiv/papers/1604/1604.02878.pdf). this model has implementations in both PyTorch and TensorFlow, but we are going to use a TensorFlow implementations, packed as a pip library by Iván de Paz Centeno (https://github.com/ipazc/mtcnn). Later on in this series, we shall see how to optimize the performance of the face detection tasks by mtcnn by use of TensorFlow serving. read more about mtcnn here.
Architecture
Our application will have the below general architecture:
For the API we shall create a simple Django app then use a PostgreSQL database for persistence.
Create Django application
lets now begin creating our application.
Assuming that we have python installed, we shall begin by installing Django.
Its always advisable to use a virtual environment for projects so as to separate development environments.
- on a preferred location on your machine, create a directory called
face_detect_apiusing the command:mkdir face_detect_api - CD into the created directory
cd face_detect_api - create and activate a virtual env for our project (I use pyenv)
pyenv virtualenv 3.6.5 face_detect_api && pyenv activate face_detect_api - Install Django and django rest framework
pip install django djangorestframework - create a new project using the command
django-admin startproject face_detect_apithencdinto the directory. The project structure should be as shown below
6. create the api application using the command python manage.py startapp api the updated project structure should be as shown below
7. Add the api app to INSTALLED_APPS in the project settings located at face_detect_api/settings.py
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'api',
]8. inside the api directory, create the file api/urls.py and copy the below contents inside
from django.conf.urls import url
from .views import NewImage
urlpatterns = [
url(r'^image/$', NewImage.as_view(), name='upload-image'),
]8. Copy the below contents to the file face_detect_api/urls.py
from django.contrib import admin
from django.urls import path, include
from django.conf.urls import url
urlpatterns = [
path('admin/', admin.site.urls),
url(r'^api/', include('../api.urls')),
]What we have done in the above two steps is configuring the urls for our django application.
9. edit the file api/views.py and add the below content inside (add a default view)
# The future is now!
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
class Image(APIView):
def post(self, request, *args, **kwargs):
return Response({"you hit the view!!"}, status=status.HTTP_200_OK)10. run the application on port 8900 (or any other port you prefer) using the command
python manage.py runserver 8900to test if everything is okay, use postman to send a POST request to the endpoint http://localhost:8900/api/image The output should be;
[
"you hit the view!!"
]
What we have accomplished so far is setting up a simple django application that is accessible externally. We now move ahead and add features to the django app to enable us to upload an image to our server.
Upload image to server
Before we get to the actual object detection task, lets first modify the view that we created to enable us to upload an image to our server. to do this, we need to add a few changes to our settings file (face_detect_api/settings.py).
Add the following entry to the settings file:
# See: https://docs.djangoproject.com/en/dev/ref/settings/#media-url
MEDIA_URL = './media/'
Next copy the below content to our view
# The future is now!
import uuid
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from django.core.files.storage import default_storage
import os
from django.conf import settings
def upload_image(request):
img = request.FILES['image']
img_extension = os.path.splitext(img.name)[-1]
return default_storage.save(settings.MEDIA_URL + str(uuid.uuid4()) + img_extension, img)
class Image(APIView):
def post(self, request, *args, **kwargs):
upload_image(request=request)
return Response({"success":"accepted"}, status=status.HTTP_202_ACCEPTED)Above, we have modified our view and added the functionality upload an image to the server. We upload the file and save it under the media directory, the file is renamed to a uuid string generated.
Now if we send a post request with an image file using postman we should get the below response
When we look at our project, the media directory should have been created, and inside, there will be the uploaded image.
Now that we are able to upload an image to our web server, lets go ahead and add functionality to detect faces.
As mention earlier in this post, we are going to use the mtcnn model to perform the face detection task. To begin, install the mtcnn, tensorflow, pillow, opencv-python and numpy pip packages
pip install mtcnn numpy opencv-python tensorflow pillow in the our view, add the following function
def detect_faces(image_path):
image = PImage.open(default_storage.open(image_path))
image = image.convert('RGB')
pixels = asarray(image)
detector = MTCNN()
# detect faces in the image
results = detector.detect_faces(pixels)
# extract the bounding box from the faces
detected_faces = list()
for result in results:
# only detect faces with a confidence of 90% and above
if result['confidence'] > 0.90:
detected_faces.append({
"face_id":uuid.uuid4(),
"confidence": result['confidence'],
"bounding_box": result['box'],
"keypoints":result['keypoints']
})
return detected_facesWith that simple function, we have completed the main bit of our fancy face detection API.
Make the final edit to out Image view to include the above function.
# The future is now!
import uuid
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from django.core.files.storage import default_storage
import os
from mtcnn.mtcnn import MTCNN
from numpy import asarray
from PIL import Image as PImage
from django.conf import settings
def upload_image(request):
img = request.FILES['image']
img_extension = os.path.splitext(img.name)[-1]
# return path to saved image
return default_storage.save(settings.MEDIA_URL + str(uuid.uuid4()) + img_extension, img)
def detect_faces(image_path):
image = PImage.open(default_storage.open(image_path))
image = image.convert('RGB')
pixels = asarray(image)
detector = MTCNN()
# detect faces in the image
results = detector.detect_faces(pixels)
# extract the bounding box from the faces
detected_faces = list()
for result in results:
# only detect faces with a confidence of 90% and above
if result['confidence'] > 0.90:
detected_faces.append({
"face_id":uuid.uuid4(),
"confidence": result['confidence'],
"bounding_box": result['box'],
"keypoints":result['keypoints']
})
return detected_faces
class Image(APIView):
def post(self, request, *args, **kwargs):
upload = upload_image(request=request)
detected_faces = detect_faces(upload)
return Response({"success":detected_faces}, status=status.HTTP_202_ACCEPTED)Now, if we post an image to our endpoint, we get the below output:
{
"success": [
{
"face_id": "fd495900-5cc5-4c99-a6f5-d48517f44c95",
"confidence": 0.9999711513519287,
"bounding_box": [
398,
298,
54,
59
],
"keypoints": {
"left_eye": [
412,
327
],
"right_eye": [
434,
317
],
"nose": [
426,
333
],
"mouth_left": [
424,
348
],
"mouth_right": [
442,
339
]
}
},
{
"face_id": "f40ed292-16bd-4b09-b83d-53990ed33c08",
"confidence": 0.9998782873153687,
"bounding_box": [
313,
184,
54,
66
],
"keypoints": {
"left_eye": [
330,
212
],
"right_eye": [
354,
206
],
"nose": [
344,
225
],
"mouth_left": [
335,
237
],
"mouth_right": [
356,
233
]
}
},
{
"face_id": "ff8e17d6-d2e9-4fca-84d1-9b28a6492962",
"confidence": 0.9996253252029419,
"bounding_box": [
247,
232,
39,
49
],
"keypoints": {
"left_eye": [
254,
254
],
"right_eye": [
272,
249
],
"nose": [
262,
262
],
"mouth_left": [
261,
273
],
"mouth_right": [
275,
269
]
}
},
{
"face_id": "45bf8b23-224a-4281-bc06-6f964e7f27dc",
"confidence": 0.9982662796974182,
"bounding_box": [
167,
245,
37,
46
],
"keypoints": {
"left_eye": [
180,
264
],
"right_eye": [
197,
261
],
"nose": [
192,
271
],
"mouth_left": [
185,
283
],
"mouth_right": [
198,
281
]
}
}
]
}A list of dictionaries with the detected faces. For each detected face, the API outputs the confidence level of the model, the bounding box of the face in the image and keypoints of facial features i.e. mouth, nose and eyes.
Up to this point, we have been able to create a simple synchronous face detection API, that takes as input, an image, then outputs the detected faces on the image.
Now its time to dockerize our application, so that development can be a bit easier. In case you are not familiar with docker, please acquaint yourself with it here.
In the root directory of the project, add the below Dockerfile
# the future is now!
FROM python:3.6
ENV LANG C.UTF-8
MAINTAINER bildad namawa "bildadnamawa@gmail.com"
RUN mkdir /django
RUN apt-get -y update
RUN apt-get install -y python python-pip python-dev
ADD requirements.txt /django/requirements.txt
RUN pip install -r /django/requirements.txt
RUN apt-get -y update && apt-get -y autoremove
WORKDIR /django
EXPOSE 8000
CMD gunicorn -b :8000 django.wsgiNext we add a docker-compose file ()because we want to run the application using docker-compose.
Add the docker-compose.yaml file at the root of the project and copy the contents below into the file.
version: '3'
services:
api:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./:/django
ports:
- "8900:8000"
From the project root directory, run the command
docker-compose upDocker will pull the necessary images and start the api container, which we can now access from the endpoint http://localhost:8900
Cool! our API is now dockerized and we can develop it from any machine without worrying about dependencies, as everything is packaged by docker when it comes to development. There is one draw back though, the state of our application is nowhere near the promised “production ready” status as alluded to in the title of this article. In order for our application to be production ready, we need to make the endpoint asynchronous and in order to do this, we shall process the images in the background.
In the next part of this series, we shall use a micro-service architecture to make our API both asynchronous and scalable. We shall also deploy the application to azure cloud. Find part 2 here
Find the code for this post in this GitHub repository
