Building a Web Application to Deploy Machine Learning Models

So we’ve built our ML model — now what? How to get out of Jupyter Notebook and into Web Apps with Flask!

If you’ve been learning Deep Learning for a while now, you may have built some really cool models using your Jupyter notebook doing a range of things, from image recognition to language translation. You probably want to let your friends try out your models (or maybe even set up a company from this), but sending them a Jupyter notebook isn’t really what you had in mind. How do you build a Web Application to deploy your Machine Learning models?

The focus of this post is building a web app around our Machine Learning model for others to try. We will go through some Web programming techniques such as HTML and Flask, as well as deploying it on the Web on a Ubuntu server on DigitalOcean.

By the end of this tutorial, other people (from all over the world) will be able to try your simple image recognition software. Your web app will look something like this page:

Once you’ve chosen a file (please choose a file below 500kb in size), you can click “Upload”. Suppose I chose the cat image that I used previously in my Notebook, I’ll be redirected to this page with my prediction:

Prediction of what is in the image

To get something up quickly, we’ll skip a few best practices along the way; but hopefully, we’ll come back to it in later posts!

Prerequisite:

This post assumes familiarity with building your own image recognition model; you may refer to the guide here to get up to speed:

Build your first Convolutional Neural Network to recognize images

Once you’ve completed the tutorial above, you should have a .h5 file with a model to classify images into 10 different classes (airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck). If you have that model file, we can begin creating a Web app around it!

If at any point you want to see the full code for what we’ll be doing in this post, please see the GitHub here.

Setting up your Web Server on DigitalOcean

[Note: as this section relies on a third party service, DigitalOcean, the instructions might change over time]

The purpose of this post is to teach you how to deploy a simple Web application on the Web for others to try. For this, we need a Web server, which is like a computer that will send information (the webpage) to other computers who request for it. We will set up a Web server on DigitalOcean, and you may use my referral link here (or just go to the website directly):

DigitalOcean - Cloud Computing, Simplicity at Scale

In that link, create an account on the right panel:

Key in your email address and password, and click the green “Create your account” button

DigitalOcean will then walk you through the steps in creating an account:

Welcome page once you’ve keyed in an email address and password if you have used a referral link

Once you click the link in the confirmation email, you will need to key in your billing information. After keying in your billing information, you will be led to this page to create your first project:

After keying in the project name and what your project is for, you can scroll down to the bottom and ignore the other questions. Click the nice blue Start banner at the bottom:

You should then reach this landing page:

Landing page for DIgitalOcean

The Web server we will need is called a “droplet” in DigitalOcean. Hover over the ‘plus’ below the heading ‘Create a Droplet’ and you’ll see this:

Now click on the blue ‘Create Droplet’ button. You should see this page:

The default (highlighted above) is Ubuntu, which is what this tutorial will be based on.

If you scroll down to Choose a plan, you can choose how powerful you want your Web server to be. The more expensive plans that you choose, you can process larger images and process them faster. I suggest picking the $10/month one, as we will need some memory to process images. However, the $5/month should be good enough to try this tutorial out:

If you scroll down further, you can also choose which physical data center you want your server to be on. I’ve chosen SF(2):

You can ignore the other options on this page for now. Scroll down to the very bottom, give your droplet a name and click the green “Create” button.

It will take a little while for your droplet to initialize, but once you’re down, you should see this as your front page:

Note that there are four numbers (157.230.156.140) which will be the IP address for our droplet. An IP address is like any web URL (like www.intuitivedeeplearning.com) so you can just enter it into your address bar. If you want a domain name for it so that others can type in English words into the address bar rather than a series of four numbers, you’ll have to purchase it and set up separately.

At this point, you’ll get an email like this:

Email from DigitalOcean

Copy your password and then go back to the DigitalOcean console. If you click the three dots on the side of your droplet, a pop-up should appear.

Pop-up when you click on the three dots on the side of your droplet

Click on “Access console”. This will open up a window which will allow you to communicate to your Web server.

Console screen

Now, type in the username and password you’ve got in your email. The username should be ‘root’. The password will not show, but if you click Cntrl-V and Enter, you should be able to get access in. The system will prompt you to change your password. Enter your current password (Cntrl-V, Enter) and then type in your new password twice.

Type in your new password for your droplet

Once you’re done, you should see a screen like this:

Now don’t be intimidated by the console. The console looks scary, but it really is a way to speak to the computer by typing very specific commands. For example, try asking what Python 3 version your server is using by typing

python3 --version

and clicking Enter. You should see something like this:

The computer then tells you the version of Python your server is using, which for me right now is Python 3.6.7.

Now, it’s time to create your first HTML file. HTML is a language which Web browsers understand, and it will turn the HTML code into the Web page you’re looking at right now. To create our files, we will need to use something like a text editor, like Notepad but on your server. We’ll be using an editor called nano for this tutorial.

Our first webpage will be called index.html which is the front page of your website. Now, type this command to your server:

nano index.html

In the console, it’ll look like this:

This command means to open up the file index.html with the editor nano. Since index.html doesn’t exist, it will create the file for you. You should see something like this on your screen now:

As the bottom suggests, this is a new file for you to type stuff in. You can type any sentence in to this Notepad-like app, for example:

This is my first Webpage!

Your console will look like this:

Now, we’ll exit out of this Notepad by clicking Cntrl-X. At the bottom, they will ask you if you wish to save your file:

You have to save your changes, so type ‘Y’. Now, they will ask you what your filename will be:

Just click Enter on your keyboard, and you’ll be done! Now, this might seem a bit less user-friendly than your standard Notepad, but just bear with it for a while.

You’ve now got a HTML file on your Webserver, all you need to do is to ‘open’ your Webserver. To do so, type this:

python3 -m http.server 80

‘80’ is the standard port which serves HTTP files. Your console should now look like this:

Now, your website is online! Go to the IP address for your droplet, which are the four numbers you see beside your droplet in DigitalOcean. You should be able to see your first webpage, which is the content you’ve written in index.html through the nano editor just now.

Congratulations — you’ve set up your first Web sever on DigitalOcean! To stop the Webpage and get back to typing commands, just press Cntrl-C. Now, we are ready to build our first Machine Learning Web app on our Web server!

Creating the Web App Backend using Flask

Your Web app can be thought of as two parts — the “backend” and the “frontend”. Think of the “backend” application as the brains behind the operations — you might not see it, but the computations are all done here. This is where we define the logic of how an user will interact with your web application, and what happens to the user’s inputs. It is also where we will run our Machine Learning model. The “frontend” on the other hand, defines what the users see, touch and experience. This is like the face of your Web application and will be what your user interacts directly with.

In this section, we will define the backend — we’ll create the “brains” first, and then we’ll see how to fit in the “face” later. In particular, we will:

1. Load our Machine Learning model;
2. Define what happens when he uploads the photo in the main homepage; and
3. Apply our Machine Learning model to the image and show the user the results in a separate “prediction” page.

In the next section, we will define how the user sees the two webpages — the homepage and the “prediction” page. But for now, let’s focus on the backend, and we’ll be using Flask as a framework to build it.

Flask is a simple web application framework that we can use to build the backend of web apps and get started quickly. In this tutorial, I will explain enough to create the web app around your Machine Learning model. Unfortunately, I will not be explaining every line of Flask code and structure; you can learn more about Flask here.

First, let’s install Flask. To do so, we first have to install pip3, which helps us to install other Python packages like Flask on our server. In your console, type

apt install python3-pip

Your console should look like this:

Once you click Enter, it will tell you the packages that will be installed. Type in “Y” to continue. Now that pip3 is installed, enter the command into your console:

pip3 install Flask

Your console should look like this:

Command to install Flask

How a successful installation looks like

While we’re installing Flask, be sure to install all the Python packages we had previously now on the server:

pip3 install keras
pip3 install tensorflow
pip3 install scikit-image

Now, we will organise our files in the server. We will createa Python file for our web backend, which we will call imgrec_webapp.py(using nano like we did earlier). We will then have two folders, one to store the uploaded images (which we will call uploads) and one to store the HTML files for your frontend (called templates). To create a folder, type mkdir folder_name into the console, where “folder_name” can be replaced by your desired name. So the two commands to type are:

mkdir uploads
mkdir templates

Now, we need to upload our model, my_cifar10_model.h5 into the server. Since the model is in my local computer, unfortunately, we can’t use the console to upload the model from my desktop. Instead, we will use Windows PowerShell. This is an app that you likely have installed already if you are using Windows:

When you’ve opened Windows Powershell, type in this command to upload your model, and be sure to replace the instructions in triangular brackets:

scp <where your model is stored>\my_cifar10_model.h5 <Droplet IP address>:~

In my case, I would type:

scp C:\Users\user\Desktop\my_cifar10_model.h5 root@157.230.156.140:~

If it is your first time connecting from Powershell, a prompt like this will appear. Simply click “yes”.

Type in your password, and once you’re done, it should look something like this:

And now your model is on the Droplet! Now, go back to the DigitalOcean console where you accessed your server previously. Alternatively, if you like the PowerShell interface, you can use PowerShell to connect to your server like this:

ssh root@<your Droplet IP>

In my case, it would be:

ssh root@157.230.156.140

And voila! You no longer need to go to DigitalOcean to get your server console. Once you’re in your console, type ls (which stands for list) to make sure that you’ve got all your files and folders:

What you should see if you’re using Powershell

What you should see if you’re using DigitalOcean console

Now that we’ve got everything set up, we can (finally) start coding our backend application. Open up the Python file you have created, imgrec_webapp.py. As usual, we will import some of the necessary functions that you will need later:

import os
from flask import Flask, request, redirect, url_for, render_template
from werkzeug.utils import secure_filename

Then, we create an instance of a Flask app which just one line of code:

app = Flask(__name__)

We’ll add more to the Flask app later. For now, let’s load our Machine Learning model that we’ve already saved in our pre-requisite tutorial my_cifar10_model.h5. We call the following functions and load the model like this:

from keras.models import load_model
from keras.backend import set_session
from skimage.transform import resize
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
print("Loading model")
global sess
sess = tf.Session()
set_session(sess)
global model
model = load_model('my_cifar10_model.h5')
global graph
graph = tf.get_default_graph()

Now that we’ve got our model running, let’s add to our Flask web app. The first thing we want to do is state what the web application will do when it hits a certain URL. To define that, we use the@app.route() function. For my main page, I will use the default URL (i.e. I won’t have any additional “suffixes” to my URL) and so my first parameter will be '/'. An user will interact with my main page in two ways:

1. Load the web page from the server into his browser
2. Upload his image and send it to the server

This requires two different methods of interacting with this page: Interaction #1 will use “GET” (a method primarily for an user to request data from a resource), while interaction #2 will use “POST” (a method primarily for an user to send data to a server to create or update a resource). The code block below therefore says that if I’m at interaction #2, save the file in my “uploads” folder and go to the “prediction” page (and pass in the parameter filename). If not, I’m at interaction #1 and just render the webpage “index.html”.

@app.route('/', methods=['GET', 'POST'])
def main_page():
    if request.method == 'POST':
        file = request.files['file']
        filename = secure_filename(file.filename)
        file.save(os.path.join('uploads', filename))
        return redirect(url_for('prediction', filename=filename))
    return render_template('index.html')

Now, we have to create a “prediction” page to go to, and define what the Web app should do when an user is at that page. We use the @app.route()function again, and say that if an user is at the /prediction/<filename> subdirectory, pass in the <filename> into the Python function that I define for that page.

In this prediction page, I will:

1. Read the image based on the filename, and store it as my_image
2. Resize the image to 32x32x3, which is the format that my model reads it in, and store it as my_image_re
3. Using the model, predict the probabilities that the image falls into the various classes, and put that under the variableprobabilities
4. Find the label and the probability of the top three most probable classes, and put them under predictions
5. Load the template of my webpage predict.html and give them the predictions in Step 4.

The five steps are reflected below:

@app.route('/prediction/<filename>')
def prediction(filename):
    #Step 1
    my_image = plt.imread(os.path.join('uploads', filename))
    #Step 2
    my_image_re = resize(my_image, (32,32,3))
    
    #Step 3
    with graph.as_default():
      set_session(sess)
      probabilities = model.predict(np.array( [my_image_re,] ))[0,:]
      print(probabilities)
      #Step 4
      number_to_class = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
      index = np.argsort(probabilities)
      predictions = {
        "class1":number_to_class[index[9]],
        "class2":number_to_class[index[8]],
        "class3":number_to_class[index[7]],
        "prob1":probabilities[index[9]],
        "prob2":probabilities[index[8]],
        "prob3":probabilities[index[7]],
      }
    #Step 5
    return render_template('predict.html', predictions=predictions)

All this should be pretty familiar, if you need a refresher, please visit the tutorial:

Build your first Convolutional Neural Network to recognize images

Lastly, we run the app with port 80, which is the standard port for accessing web pages. Type this at the end of your Python file:

app.run(host='0.0.0.0', port=80)

Now we have the backbones of our Web application. You might have noticed that we are still missing a few different pieces though. In particular, we’ve called on the HTML web pages like index.html and predict.html but we’ve not built them yet! We will do so in the next section.

Do also note that in this bare-bones tutorial, we’ve skipped many good practices. You may want to try incorporating some of these practices if you know how to (or Google it!):

• Creating our Web app in an Anaconda environment (like we did in Anaconda Navigator previously)
• Catching errors such as someone not uploading any file, or with a blank filename
• Checking link extensions that people are uploading ‘.jpg’ files

Creating the Web App Frontend using HTML

When a user interacts with your web application, they do so via the web pages, which is what we will be building in this section. Go into the templates folder by typing in the console:

cd templates

Create the two template files we need, index.html and predict.html. If you type ls you should see those two files under templates:

We use a language called HyperText Markup Language (HTML), which just tells the browser how to render the content that you have. To generalise it a little bit, we tag the content with tags like “header” or “paragraph” and then we tell the browser how to display content with those type of tags. For example, if I want to tag the words “Hello” as a “header 1” type, I’ll write:

<h1>Hello</h1>

Usually, when you start the tag (i.e. <h1>), you have to close the tag as well to show where it ends (i.e. </h1>). We’ll learn as we go, so let’s start with index.html. To remind you how the page will look like, we are trying to build something like this:

The first thing we start index.html with is this line:

<!doctype html>

This tells the browser what kind of document type to expect (i.e. HTML). We then begin the document with a<html> tag, which we will close it at the very end. Now, we also define the head of the document, which contains the “meta-data” of the document. Here, I’ve defined the title and I’ve linked a Bootstrap stylesheet, where other people have already defined styles for the various tags. I can therefore leave large amounts of the styling to what others have coded for us, unless there are some aspects I need to customise.

<html>
<head>
    <title>Classify your Image!</title>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
</head>

Now, we define the body:

<body style="background-color:black">
<div id="maintext" style="position:absolute; margin:auto; width:700px; height:200px; text-align:center; top:0; bottom: 0; left: 0; right: 0;">  
  <h1 class="text-light mb-2"> Upload your image to be classified! </h1>
  <h4 class="text-light mb-5">(Please upload images less than 500kb in size)</h4>
  <form method=post enctype=multipart/form-data>
     <p class="text-light">
        <input type=file name=file>
        <input type=submit value=Upload>
     </p>
  </form>
</div>
</body>
</html>

First, I tell HTML that I want a background color of black. Then, I have a div container so that I can contain the following content together. I want the container to be in the middle of the page with a certain height and width and margins, so I define that in the style. Within my div container, I have three main content pieces:

1. Large header (h1) that says “Upload your image to be classified!” (Note: mb-2 in the style just says add a margin of “medium size” at the bottom)
2. Small header (h4) that says “(Please upload images less than 500kb in size)”
3. A “form” that allows me to upload a file and then submit that form. We use the POST method to submit the form (recall how we defined the backend?)

Once I’ve defined these three content pieces within my div container, I close the body and html tags and I’m done! Do note that it is good practice to re-organizing your code into CSS files and HTML files so that we can separate the style from the content — for the purpose of this tutorial, we’ve taken some shortcuts!

Now, we move on to the predict.html page. Remember how the page is supposed to look like:

For the HTML code, the head is still the same as index.html, but there are a few new elements introduced in the body:

<!doctype html>
<html>
<head>
    <title>Your Prediction!</title>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
</head>
<body style="background-color:black">
<div id="maintext" style="position:absolute; margin:auto; width:700px; height:300px; text-align:center; top:0; bottom: 0; left: 0; right: 0;">
  <h1 class="text-light mb-5"> Your Prediction </h1>
  <p class="text-light"> My algorithm gave this prediction: </p>
  <table class="table-bordered text-light" style="width:100%">
  <tr>
    <th>Rank</th>
    <th>Class</th>
    <th>Probability</th>
  </tr>
  <tr>
    <td>Most Likely:</td>
    <td>{{ predictions.class1 }}</td>
    <td>{{ predictions.prob1 }}</td>
  </tr>
  <tr>
    <td>2nd Most Likely:</td>
    <td>{{ predictions.class2 }}</td>                                                                                                                                       <td>{{ predictions.prob2 }}</td>                                                                                                                                      </tr>                                                                                                                                                                   <tr>
    <td>3rd Most Likely:</td>
    <td>{{ predictions.class3 }}</td>
    <td>{{ predictions.prob3 }}</td>
  </tr>
  </table>
<a href="{{ url_for('main_page') }}"><button class="button mt-3">Try again?</button></a>
</div>
</body>
</html>

In my div container (which is the same as in index.html), I now have the following content:

1. A header (h1) that says “Your Prediction”
2. A paragraph (p) that says “My algorithm gave this prediction:”
3. A table with three columns and four rows. Each row in the table is tagged with <tr> . For each row, the headers are tagged with <th> and the text for each column is tagged with<td>. Note that the content for my predictions are not fixed — it depends on what my model gives me, which depends on what the user uploaded. This is where I need to interact with the backend. If you recall, in my prediction backend, the last line of the function is return render_template(‘predict.html’, predictions=predictions). This means that when I rendered the “predict.html”, I pass to my frontend the variables of predictions, and these variables will appear in my frontend wherever I call them using the double curly braces {{ ... }}. Therefore, {{ predictions.class1 }} will be automatically filled in with whatever the model predicted as the most probable class.
4. A button to link me back to the main page. I use {{ url_for(‘main_page’) }} because main_page is the name of the function in my backend that defined the main page. Therefore, whatever the URL is will be filled in automatically here.

And there we have it! Again, it is good practice to re-organizing your code into CSS files and HTML files so that we can separate the style from the content. This way, if we want to consistently change the style of the div for example, we don’t need to go to both index.html and predict.html to change it.

Run your Web Application!

Now, all there is left to do is to run the Web application. Go back to the console and type this:

python3 imgrec_webapp.py

You’ll see a whole bunch of warning messages, but you can ignore them. The most important thing is that at the end, you see this screen:

Now, you can enter your droplet’s IP address into your browser and click Enter.

Replace this with the IP address of your droplet

You should see your application up and running, and you should be able to upload a small image and get a prediction! Do note that the first image will always take a lot longer, so be patient! It will be faster after that!

Consolidated Summary: In this post, we’ve written a web application around our image recognition model by:

• Setting up on the cloud (DigitalOcean)
• Creating the Web App Backend using Flask
• Creating the Web App Frontend using HTML

Congratulations! You’ve built your very first web application. We are still quite far away from building a production-level web application, and we have not incorporated many necessary practices if we are to scale this up. But at least we have some semblance of a Web application running, and we can incorporate these practices to our base framework.

What’s Next: In our next Coding Companion Part 3 (link to be released), we will explore how to code up our own Recurrent Neural Networks (RNNs) to deal with natural language! Be sure to get an intuitive understanding of RNNs here:

Intuitive Deep Learning Part 3: RNNs for Natural Language Processing