Serverless Image Recognition with KNIME and Amazon AWS Rekognition

KNIME output with Amazon Rekognition

Building an accurate image recognition engine with deep learning tools can be a difficult undertaking. You may require tens of thousands (or millions) of curated, tagged images to get an accurate model and a powerful enough server to run the deep learning model.

Well, luckily for us, all major cloud vendors have solved this challenge by providing API’s for their machine learning image recognition services. Google has Cloud Vision, Microsoft Azure has Computer Vision and Amazon has Rekogniton.

The approach I take with my tutorial using Amazon can be adapted and used for any of the services above. At the end of the day, an image is uploaded to cloud storage, an API call issued and a JSON output is returned with the images classification.

This is a quick and dirty tutorial, I’m sure you could improve this workflow somewhat if you’re going to place it into a production environment.

Things you’ll need

Python (I’ll be using Anaconda)

An Amazon AWS Account

KNIME Analytics Platform

We’ll also be installing a Python module, boto3 and the AWS CLI as part of this tutorial so we can interact with AWS’s services within the Python script.

Installing the CLI and Python module

Assuming you’ve signed up for your Amazon Web Services (AWS) account and you’ve started a free trial. We need to install the AWS Command Line Interface (CLI) and as I’ll be using Python, the Amazon Python library boto3.

I’m using an Apple Mac to create this tutorial along with Anaconda as my Python environment. With a Mac you can easily install the CLI by running in the Terminal;

pip install awscli

For Windows users, you can download the CLI from Amazon.

Once installed we need to configure the CLI with our Amazon credentials and default region.

  1. Lets create some credentials through IAM for our CLI to use. Open the AWS Console and Navigate to IAM or click this link.
  2. Select Users on the left hand menu bar and select Add user.
Adding a new IAM user for the Amazon CLI

3. Specify a username, something like AWS_CLI so you can easily identify it later if you need to and tick the Programmatic access checkbox next to Access type.

4. We’ll need to provide access to Amazon services for this new user. For speed in this tutorial, I’m going to give my AWS_CLI Administrator Access. This provides the user with full access to all AWS services. If you wish you can provide more granular controls.

If you’d rather, you can just specify access to S3 and Rekognition

5. We can skip tags and then create the user. Once created you’re provided with an Access key ID and a Secret access key. We’ll configure the CLI tool with our credentials.

We do this by running;

aws configure

This will prompt us for our AWS Access Key ID, AWS Secret Access Key, Default region name, Default output format. Enter the information above along with your default region. You can leave Default output format blank.

You can find a list of your local regions here;

AWS Regions and Endpoints — Amazon Web Services

6. Install the AWS Python Library boto3. You can do this either by running;

pip install boto3

Or if you’re using Anaconda (which I highly recommend), you can use the Anaconda package manager or install using conda;

conda install -c anaconda boto3

Configuring KNIME to leverage AWS Rekognition

KNIME workflow incorporating AWS Rekognition via Python and boto3

You can download the KNIME workflow at the link below;

https://www.dropbox.com/s/35puhs75epj5ova/Image%20Recognition.knar.knwf?dl=0

To configure the KNIME workflow to work with your data;

  1. Configure the List Files node with a folder on your local hard drive that contains images you wish to identify.
  2. Configure the Upload node with a unique S3 bucket name. You’ll want to place the files into a folder within the bucket named images as the folder name is hard coded in the Python script. For example;
/myrekognitionbucket/images/

The python script leverages KNIME variables and importantly uses a Variable Loop node to loop through the files in the folder, running the Python script each time with the new filename contained in the directory. If you’re not familiar with the Loop node, I highly recommend you look into it and have a play.

The Python code looks like this;

from pandas import DataFrame
import pandas as pd
import boto3
fileName='images/'+(flow_variables['Path_Arr[3]'])
bucket=flow_variables['Path_Arr[1]']
client=boto3.client('rekognition')
response = client.detect_labels(Image={'S3Object':{'Bucket':bucket,'Name':fileName}})
data = [str(response)]
df = pd.DataFrame(data)
print(df)
output_table = df

By importing boto3, we can then call boto3.client(‘rekognition’) to start interacting with the Rekognition service.

This data is then output in JSON format and turned into a table.

In the filter confidence metanode we pivot the data, remove unnecessary columns and filter out results that fall below a certain confidence threshold as Rekognition provides us with confidence values. This helps improve the quality of results displayed.

The final step we join the AWS Rekognition output with the original images for presentation, as you can see below;

If you have any questions or ideas about this workflow, or any of my other tutorials, please feel free to reach out to me on Twitter or LinkedIn. I love the helpful KNIME community, I hope you can take this, improve on it and use it in your next project.