Part I — Deploying a Keras Computer Vision Model to AWS using an AWS EC2 Instance (Web App & API)
Learn how to turn your Computer Vision (CV) Deep Learning Keras or TensorFlow Model into API or Web App.
I’ve created two very popular (~15,000 copies) Computer Vision courses on Udemy and since its inception, one of the most prevalent questions has been:
How do I take my OpenCV or Keras model and deploy it on a mobile app or web?
My two courses cover OpenCV and Deep Learning using Keras in Python.
Computer Vision Intro™ OpenCV4 in Python with Deep Learning
Deep Learning Computer Vision™ CNN, OpenCV, YOLO, SSD & GANs
I’ve finally made a simple (though a bit long) tutorial showing you how to use an all Python system to do just this! There’s no messy javascript or native mobile coding involved.
Before we begin, let’s discuss three (3) main ways you can do this:
1. Create an API, which allows any other application (whether it’s an iOS, Android or within any other software) to access it via the API protocol. APIs are Application Programming Interfaces which facilitate access to complex external applications via a simplified interface. In our example, we’ll be using the RESTful API protocol which allows us to run our Python CV via the web.
2. Create a Web App, this is basically the same as an API, except we’ll be allowing users to access or run our CV app via any web browser.
3. Run the CV code natively on another platform. This is the more difficult option is it requires you to re-code your Python CV code into another language. This is far more difficult and requires you learning an entirely new language e.g. Java if making an Android App. But the local access reduces the need to send data over the web and can perform real-time operations. This method won’t be shown in this tutorial.
Introducing Flask
Before we begin installing Flask and setting up everything on AWS, let me first explain what exactly is Flask, and why we need it.
Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries. It has no database abstraction layer, form validation, or any other components where pre-existing third-party libraries provide common functions. This allows us to easily build simple APIs without much effort.
Step 1: Installing Flask
Assuming you have TensorFlow, Keras & OpenCV installed — activate that environment. If you don’t have any of these packages installed, you can do so by following these instructions (Ubuntu only, but it should be similar for Windows and Mac):
sudo apt update #Not needed if using Mac or Windows
sudo apt install python3-pip #Not needed if using Mac or Windows, install Anaconda instead
sudo pip3 install tensorflow
sudo pip3 install keras
sudo pip3 install flaskNote: As of April 2019, sudo pip3 install opencv-python appears to be broken on Ubuntu, if importing cv2 fails, run sudo apt install python3-opencv
sudo pip3 install opencv-python
sudo apt install python3-opencvsource activate cv or whatever your environment is named
pip install flask
That’s it, Flask is ready to go!
Let’s run a simple Flask test.
Step 2: Installing a code editor since we won’t be using Ipython Notebooks. I recommend Sublime, but VSCode, Atom, Spyder, and PyCharm are all great!
If using Windows or Mac, visit the Sublime website to get the download link — https://www.sublimetext.com/download
Ubuntu/VM users can bring up the Ubuntu software app, search for Sublime and install it from there.
Open up your chosen Text/Code Editor and copy the code below into it and save it in an easy to find directory, e.g.
/home/deeplearningcv/MyFlaskProjects
This is code for your first simple Flask app, the ‘hello world’ app.
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello World!"
if __name__ == "__main__":
app.run()Once your code is saved in the path, use Terminal or Command Prompt and navigate to the folder where you saved your helloworld.py file.
Upon executing this code, you’ll see the following:
This means your server is running successfully and you can now go to your web browser and view your hello world project!
You should see this if you enter 127.0.0.1:5000 in your browser. This means you Flask server is pushing/serving the text "Hello World!" to your localhost on Port 5000.
Remember to press CTRL+C in the Terminal to exit
Congratulations! Now you’re ready to start testing your Computer Vision API locally!
Our Flask WebApp Template Code
In the preceeding work, we created a simple Hello World app. We’re now going create a simple App that does the following:
- Allows you to upload an image
- Uses OpenCV and Keras to do some operations on the image
- Returns the outputs of our operations to the user (i.e. the dominant color and whether the image contains a Cat or Dog)
We’ll be making a simple App that users OpenCV to find the dominant color (Red vs Green vs Blue) and determines whether the animal in the picture is a Cat or Dog.
The Web App We’re About To Make!
Download the code here (if not already downloaded above)
The code for our web app is as follows:
import os
from flask import Flask, flash, request, redirect, url_for, jsonify
from werkzeug.utils import secure_filename
import cv2
import keras
import numpy as np
from keras.models import load_model
from keras import backend as KUPLOAD_FOLDER = './uploads/'
ALLOWED_EXTENSIONS = set(['png', 'jpg', 'jpeg'])
DEBUG = True
app = Flask(__name__)
app.config.from_object(__name__)
app.config['SECRET_KEY'] = '7d441f27d441f27567d441f2b6176a'
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDERdef allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS@app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
image = cv2.imread(os.path.dirname(os.path.realpath(__file__))+"/uploads/"+filename)
color_result = getDominantColor(image)
result = catOrDog(image)
redirect(url_for('upload_file',filename=filename))
return '''
<!doctype html>
<title>Results</title>
<h1>Image contains a - '''+result+'''</h1>
<h2>Dominant color is - '''+color_result+'''</h2>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''def catOrDog(image):
'''Determines if the image contains a cat or dog'''
classifier = load_model('./models/cats_vs_dogs_V1.h5')
image = cv2.resize(image, (150,150), interpolation = cv2.INTER_AREA)
image = image.reshape(1,150,150,3)
res = str(classifier.predict_classes(image, 1, verbose = 0)[0][0])
print(res)
print(type(res))
if res == "0":
res = "Cat"
else:
res = "Dog"
K.clear_session()
return resdef getDominantColor(image):
'''returns the dominate color among Blue, Green and Reds in the image '''
B, G, R = cv2.split(image)
B, G, R = np.sum(B), np.sum(G), np.sum(R)
color_sums = [B,G,R]
color_values = {"0": "Blue", "1":"Green", "2": "Red"}
return color_values[str(np.argmax(color_sums))]
if __name__ == "__main__":
app.run()
Running our web app in Terminal (or Command Prompt, it should be exactly the same).
Brief Code Description:
Lines 1 to 8: Importing our flask functions and other relevant functions/Libraries we’ll be using in this program such as werkzeug, keras, numpy and opencv
Line 10 to 16: Defining our paths, allowed files and setting our Flask Parameters. Look up the Flask documentation to better understand
Line 18 to 10: Our ‘allowed files function’, it simply checks the extension of the selected file to ensure only images are uploaded.
Line 22: The route() function of Flask. It is a decorator that tells the Application which URL should call the associated function.
Line 23 to 58: Our main function, it results to both GET or POST Requests. These are HTTP methods that form the basis of data communication over the internet. GET Sends data in unencrypted form to the server. Most common method. POS is used to send HTML form data to the server. Data received by POST method is not cached by the server. We’re using an unconventional method of serving the HTML to our client Typically Flask apps used templates stored in a Templates folder which contains our HTML. However, this is a simple web app and it’s better for your understanding if we server the HTML like this. Note we have two blocks of code, one is the default HTML used prompting the user to upload an image. The second block sends the response to the user.
Line 60 to 73: This is our Cats vs Dogs function that takes an image and outputs a string stating which animal is found in the image, either “Cat” or “Dog”.
Line 75 to 81: This function sums all the Blue, Green and Red color components of an image and returns the color with the largest sum.
Line 83 to 84: Our main code that runs the Flask app by calling the app.run() function.
Our Folder Setup:
MyFlaskProjects/
------Models/ (where our Keras catsvsdogs.h5 is stored)
------Uploads/ (where our uploaded files are stored)
------webapp.py
Feel free to experiment with different images! Let’s now move on to a variation of this code that acts as a standalone API.
Our Flask API Template Code
In the preceding chapter, we created a Web App that’s accessible via a web browser. Pretty cool! but what if we wanted to call this API from different Apps e.g. a Native Android or iOS App?
Let’s turn it into RESTful API that returns simple JSON responses encapsulating the results.
NOTE: RESTful API is an application program interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data.
Step 1: Firstly, install Postman to test our API
- Windows/Max — Download and install here
- Ubuntu Users — Launch Ubuntu Software and Install
Step 2: Our Flask API Code
import os
from flask import Flask, flash, request, redirect, url_for, jsonify
from werkzeug.utils import secure_filename
import cv2
import numpy as np
import keras
from keras.models import load_model
from keras import backend as KUPLOAD_FOLDER = './uploads/'
ALLOWED_EXTENSIONS = set(['png', 'jpg', 'jpeg'])
DEBUG = True
app = Flask(__name__)
app.config.from_object(__name__)
app.config['SECRET_KEY'] = '7d441f27d441f27567d441f2b6176a'
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDERdef allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS@app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
# check if the post request has the file part
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
# if user does not select file, browser also
# submit an empty part without filename
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
image = cv2.imread(os.path.dirname(os.path.realpath(__file__))+"/uploads/"+filename)
color_result = getDominantColor(image)
dogOrCat = catOrDog(image)
#return redirect(url_for('upload_file',filename=filename)), jsonify({"key":
return jsonify({"MainColor": color_result, "catOrDog": dogOrCat} )
return '''
<!doctype html>
<title>API</title>
<h1>API Running Successfully</h1>'''def catOrDog(image):
'''Determines if the image contains a cat or dog'''
classifier = load_model('./models/cats_vs_dogs_V1.h5')
image = cv2.resize(image, (150,150), interpolation = cv2.INTER_AREA)
image = image.reshape(1,150,150,3)
res = str(classifier.predict_classes(image, 1, verbose = 0)[0][0])
print(res)
print(type(res))
if res == "0":
res = "Cat"
else:
res = "Dog"
K.clear_session()
return resdef getDominantColor(image):
'''returns the dominate color among Blue, Green and Reds in the image '''
B, G, R = cv2.split(image)
B, G, R = np.sum(B), np.sum(G), np.sum(R)
color_sums = [B,G,R]
color_values = {"0": "Blue", "1":"Green", "2": "Red"}
return color_values[str(np.argmax(color_sums))]
if __name__ == "__main__":
app.run()
Using Postman (see image above and the corresponding numbered steps below:
- Change the protocol to POST
- Enter the local host address: http://127.0.0.1:5000/
- Change Tab to Body
- Select the form-data radio button
- From the drop-down, select Key type to be file
- For Value, select one of our test images
- Click send to send our image to our API
- Our response will be shown in the window below.
The output is JSON file containing:
{
"MainColor": "Red",
"catOrDog": "Cat"
}The cool thing about using Postman is that we can generate the code to call this API in several different languages:
See blue box to bring up the code box:
Code Generator
Now that you’ve got your Flask API and Web App working, let’s look at deploying this on AWS using an EC2 Instance.
