Using AWS Rekognition with Thumbor

In this post, we’ll take a look at the technology of image cropping at Kono and introduce our open source tool to automate the photo cropping process.

Introduction

In Kono, we have a main image to represent every single article in a magazine. To put into containers in different sizes, we crop article’s main image to various aspect ratios so that the image will not be distorted and introduce any visual artifacts. In the past, we cropped around the center of image, regardless of the human faces in the photo. The cropped result is not satisfied and make article less attractive. One year ago, we set out to change that.

Thumbor

Thumbor is an open source, smart imaging service which enables cropping, resizing image via HTTP API call. It also features smart detection of important points in the image for better cropping and resizing.

Kono uses Thumbor to crop article’s image to change aspect ratio for different devices. To better decide what region to crop rather than “center” cropping, Thumbor’s out-of-the-box face detection and feature recognition can get the “focal point” of the image and produce better cropped result.

Use face detection algorithm to decide where to crop the image is way better than “center” crop. Left image is copped centrally, the right hand side is face detection based cropping.

To enable smart detection algorithm cropping, just tell Thumbor what detectors it should run in the original image by adding them to the thumbor.conf file.

DETECTORS = [
    'thumbor.detectors.face_detector',
    'thumbor.detectors.feature_detector',
]

If a face is detected, the feature detection will not be run.

AWS Rekognition

Amazon Rekognition is a service that uses deep neural network model to add image analysis to your applications. It can easily locate faces within images and analyze face attributes. For non-frontal faces, AWS Rekognition also has decent result.

AWS Rekognition is very effective for frontal faces.

For non-frontal faces, AWS Rekognition also performs pretty well.

Our goal is to replace Thumbor’s out-of-the-box face detector with AWS Rekognition which has outperformed result generally.

Custom detector in Thumbor

Thumbor has designed its system to allow implementing tailor-made custom detector. The custom detector should inherit from the BaseDetector and implements a detect method that return “focal points” it can find. In the detect method, we call AWS Rekognition API to get all the bounding boxes of human faces in the image, and return the geometric center of each bounding box as the focal point to Thumbor. That’s it!

And we are pleased to open source the custom detector from Kono: thumbor_rekognition. Installation is easy.

pip install thumbor_rekognition

Next, go edit the thumbor.conf to tell Thumbor to import this custom detector.

DETECTORS = [
    # Do not use out-of-box face detector
    # 'thumbor.detectors.face_detector',

    # Include thumbor_rekognition
    'thumbor_rekognition',
    'thumbor.detectors.feature_detector'
]

The AWS Rekognition authentication is handled by boto3, please see the document. To specify what region to use, add them to the thumbor.conf .

# AWS region for the Rekognition service
REKOGNITION_REGION = 'us-west-2'

Conclusion

With Thumbor and AWS Rekognition, we can crop image with confidence, make it easy to ensure an ideal crop. We hope thumbor_rekognition can serve as an example of how to implement a custom detector in Thumbor and make cropping more easily and intelligently. Finally, if you want to contribute, please visit https://github.com/yu-liang-kono/thumbor_rekognition

Thanks!