Tutorial — Image Classifier using Resnet50 Deep Learning model (Python Flask in Azure)

A simple Image classifier App to demonstrate the usage of Resnet50 Deep Learning Model to predict input image. This application is developed in python Flask framework and deployed in Azure. At the end of this article you will learn how to develop a simple python Flask app that uses Keras Python based Deep Learning library to predict an image and deployed in Azure. At the end of this tutorial you will learn:

• Using Keras to predict image using Resnet50 Deep Learning Model
• Use Python Flask Framework to develop a Image Classifier App
• Deploy the Image Classifier App in Azure

Step 1 — Prepare the flask app

In my earlier article I had covered the basics of running a simple flask app. The same can be found here.

Step 2 — Simple Image Classifier app

The Image Classifier App is going to use Keras Deep Learning library for the image classification. The first step is to create a Resnet50 Deep learning model trained using imagenet.

What is Resnet50 ?
Resnet is short name for Residual Network that supports Residual Learning. The 50 indicates the number of layers that it has. So Resnet50 stands for Residual Network with 50 layers.

Deep convolutional networks have led to number of breakthroughs for image classification. In general the trend is to go more deeper number of layers to solve complex tasks and to increase the classification and recognition accuracy. But as we go deeper with the neural networks the accuracy starts saturating and then degrades also. Residual training tries to solve this problem.

In a general deep convolutional neural network, many layers are stacked and trained to the task at hand. In residual learning, instead of trying to learn some features, try to learn some residual. Residual can be simply understood as subtraction of feature learned from input of that layer. ResNet does this using shortcut connections (directly connecting input of nth layer to some (n+x)th layer. It has proved that training this form of networks is easier than training simple deep convolutional neural networks and also the problem of degrading accuracy is resolved.

If you are interested in learning more about the residual networks, you can read more here.

What is imagenet ?
The ImageNet project is a large visual database designed for use in visual object recognition software research. More than 14 million images have been hand-annotated by the project to indicate what objects are pictured and in at least one million of the images, bounding boxes are also provided.

Step 3 — Making Prediction using Resnet50 Model

Why are we using Resnet50 Model ?

Resnet50 is a pre-trained Deep learning model. A pre-trained model is trained on a different task than the task at hand and provides a good starting point since the features learned on the old task are useful for the new task. Deep Neural networks have a large number of unknown parameters. To find all the unknown parameters would require lots of data (in millions). It is very difficult to get such large labelled dataset. Instead we leverage models that have already being trained on very large amounts of data for difficult task with thousands of classes. Many Research groups share the models they have trained for competitions like ILSVRC. So instead we use these models as a starting point for our training process, instead of training the own model from scratch. There are different models like AlexNet, VGGNet, Resnet etc. These networks are trained for classifying images into one of the 1000 categories or classes. Keras comes bundled with these models and so we are using one of these models in this sample.

In this step we shall build a simple prediction application that uses Resnet50 model in Keras.

Loading the Resnet50 models in Keras:
# Keras

from keras.applications.imagenet_utils import preprocess_input, decode_predictions
from keras.models import load_model
from keras.preprocessing import image
from keras.applications.resnet50 import ResNet50
model = ResNet50(weights=’imagenet’)

In the above code, first import the python module containing the model. Then load the model architecture and the imagenet weights for the network.

Loading the Image for Prediction:
Once the Model is loaded, we load and pre-process the image for prediction. Keras provides support for importing images and performing pre-processing before prediction:

1. Load an image in PIL format ( width x height x channels )
   original = image.load_img(img_path, target_size=(224, 224))
2. Convert the image from PIL format to NumPy format ( height x width x channels )
   numpy_image = image.img_to_array(original)
3. Convert the image from NumPy format to Batch format (batchsize x height x weight x channels) which will add an extra dimension.
   image_batch = np.expand_dim(numpy_image, axis=0)

Predicting the Image:
Once the image is loaded in the right format, the next step is to feed to the network and get the right predictions.

The image that was setup in the previous step should be normalized by subtracting the mean of the ImageNet data. This is required since the images were trained after pre-processing. The steps involved are as follows:

1. Pre-process the image by subtracting the mean value from each channel of the images in the batch. Mean is the array of elements obtained by the average of R,G,B pixels of all images obtained from imagenet. This is done using the pre-process_input function.
   processed_image = preprocess_input(image_batch, mode=’caffe’)
2. Get the classification result by calling the model.predict function which returns a tensor of dimension (batchsize x 1000).
   preds = model.predict(processed_image)
3. Convert the result to human readable labels, using the method decode_predictions method. This method takes the classification result, sorts it according to the prediction and returns the class name.
   pred_class = decode_predictions(preds, top=1)

So the above code is what we need to classify an image using the Resnet50 model. We would abstract it in a function model_predict and will use it to predict the image that the user is choosing.

Step 4 — Wiring up the Flask App to make predictions

Now that we have the deep learning model setup for prediction, in this step we would wire up the Python flask app to take user input (image) and make prediction. In this step it is about wiring up the HTML to take the user input and make prediction using the model_predict function.

The home page would have a HTML form to choose an image. Once the image is selected, it would be displayed below and a predict button is shown at the bottom. The UI would look like this:

On clicking the Predict button, the earlier defined model_predict method should be invoked and the result will be displayed. For this purpose we wire up some javascript.

On clicking the Choose button, we load the image from the form into the imagePreview div.

Then on clicking the predict button we post back to a different end point that would call the model_predict function. Through Flask we expose another end point /predict which gets called on clicking the Post button. The predict end point is passed with the selected image file as the data.

The /predict end point will access the image from the request, stores it locally in the /uploads folder and invokes the model_predict to output the prediction from the Resnet50 DeepLearning model defined earlier.

That’s it. At this point you have a simple image classifier App that uses Resnet50 model to make a prediction. Following would the result on clicking the predict button.

Step 5 — Deploy the Flask App to Flask

To deploy the Flask App to Azure, let us create a Azure Web App with Linux as the OS.

Once the deployment is completed, the web app is available at — http://<app_name>.azurewebsites.net

Click the Deployment Centre and select the Local Git as the option for deployment:

In the next step select K App Service build service for deployment. Once you select Finish, a local Git repository url will be generated. The local Git repository would be off the form —
https://<azure app>.scm.azurewebsites.net:443/imageclassifier-flask.git.

Once you finish the Finish you would get a git repository url and also you have to set the user credentials under the FTP/Credentials in the Deployment Centre. Remember you would need the User Credentials while uploading the code from your git repository to Azure deployment Git repository.

Now go to the local git terminal. From your local git terminal add azure remote to your local git repository. First add the remote Url to the git repository you got from the deployment centre.

git remote add azure-dep https://<username>@<azure app>.scm.azurewebsites.net:443/imageclassifier-flask.git

Where <username> is the user name from the User Credentials that you have set in the Deployment Centre.
Where <aureApp> is the Azure Web Service App that you had provisioned for deploying this Flask App.

Now make any changes and commit your changes to the local git (git add . And git commit -m “My Changes”)

And you push your changes “azure-dep” to deploy with the following command:

git push azure-dep master

For the first time you would be prompted by credentials using the Git credentials manager. Make sure you that you enter the credentials you have configured in the Deployment Centre / User Credentials and not the credentials you use it for log into Azure portal.

After some time you will see following :

C:\TechSamples\GitRepos\imageclassifier>git push azure-dep master
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 372 bytes | 372.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Deploy Async
remote: Updating branch ‘master’.
remote: Updating submodules.
remote: Preparing deployment for commit id ‘05dd65163d’.
remote: Oryx-Build: Running kudu sync…
remote: Kudu sync from: ‘/home/site/repository’ to: ‘/home/site/wwwroot’
remote: Ignoring: .deployment
remote: Copying file: ‘app.py’
remote: Deleting file: ‘app-backup.py’
remote: Ignoring: .git
remote: Running oryx build…
remote: Build orchestrated by Microsoft Oryx, https://github.com/Microsoft/Oryx
remote: You can report issues at https://github.com/Microsoft/Oryx/issues
remote:
remote: Oryx Version : 0.2.20190820.2, Commit: 450179ca187b5b9080175bb25f5b22466c63614b
remote: Build Operation ID: |o+ctvHS4LmU=.89e56537_
remote: Repository Commit : 05dd65163da9c9ac1e09533cbfb5201c9f6a83bc
remote: Source directory : /home/site/wwwroot
remote: Destination directory: /home/site/wwwroot
remote: Python Version: /opt/python/3.7.4/bin/python3
remote: Python Virtual Environment: antenv
remote: Creating virtual environment …
remote: Activating virtual environment …
remote: Upgrading pip…
remote: Requirement already up-to-date: pip in ./antenv/lib/python3.7/site-packages (19.2.3)
remote: Done in 7 sec(s).
remote: Running pip install…
remote: Done running pip install.
remote: Removing existing manifest file
remote: Creating a manifest file…
remote: Manifest file created.
remote: Done in 21 sec(s).
remote: Running post deployment command(s)…
remote: Deployment successful.
remote: App container will begin restart within 10 seconds.
remote: Deployment Logs : ‘https://imageclassifier-flask.scm.azurewebsites.net/newui/jsonviewer?view_url=/api/deployments/05dd65163da9c9ac1e09533cbfb5201c9f6a83bc/log'
To https://imageclassifier-flask.scm.azurewebsites.net:443/imageclassifier-flask.git
656a4c1..05dd651 master -> master

At this point your deployment is completed. You can check the status of deployment in the Deployment Center

You can browse to the deployed Azure Flask based application for Image classification here — https://imageclassifier-flask.azurewebsites.net/

This entire source code is available in the following Git hub repository. (https://github.com/venknar/imageclassifier)

In summary this sample demonstrates developing a simple ML App for image classification using Resnet 50 Deep Learning Model. It also takes you through on how to develop this app in Python Flask Framework and deploy the same to Azure Web Service.