How To Deploy Digit Recognition Model Into Drawing App

Deploy your Python digit recognition neural network model into web HTML5 drawing app using flask

Photo by Christopher Gower on Unsplash

Hello there! In this article I am going to guide you how to create AI to detect handwritten digit and create web drawing app as user interface. The intuition of making this article is because I found many tutorial online that teach me how to create an AI model but didn’t until creating an app. I hope this article would be helpful for you that looking for comprehensive guide.

Application Demonstration

Why Create an Application ?

The final result of artificial intelligence should be a product. In this case, our product is a drawing app with a features to detect the handwritten digit using the AI model itself. This would be unrealistic project if the final product we have use terminal as interface, so making an application would be a valid portfolio.

Application Workflow

We are going to build both backend and frontend application. The best way to explain complex process is through visualization. I made these workflow diagram for better communication. We will use javascript in our client side for building drawing app and using python in our backend for processing and prediction.

Application workflow diagram

Backend workflow diagram

Tensorflow, Flask, Opencv, sklearn and numpy is all of python libraries we are needed in this project. I recommend to use virtual environment such as venv or virtualenv because we may have different version of library in our computer for each different projects.

STEP #1: Create Model

First step is to create classification model using Keras. MNIST handwritten is a dataset commonly used in many tutorials to get started with neural network. Using sequential API, you can easily create neural network (NN). There is no need to create complex neural network, but if you want to get maximum performance, then use Convolution Neural Network (CNN) which is good to classify images.

Create model for MNIST handwritten digit classification

In the sequential NN, I used Flatten layer with input shape of (28, 28, ). Be sure to add comma in the last tuple, NOT just (28, 28). Because the (28, 28) is for the image size, and the comma in last tuple is going to be the amount of our training data. If you try print(x_train.shape) the shape would be (60000, 28, 28) because we have 60000 training image with a size of 28 x 28.

You can copy and paste the code above to google colab, jupyter notebook, or pycharm. The model.save(filename.h5) is going to store the model in the same directory this program ran, but if you use google colab you should download the .h5 model. You can name the file anything you want but it should end with .h5 file extension. If you are happy with the model performance, then you can go to the next step.

STEP #2: Create Flask App

Create flask app to handle user requests from the front end and render html using template engine. Create new folder app and create new file inside named main.py. Also move your model .h5 file into the app folder.

After receiving base64 data from front end, we need to convert the image from 3 channel to 1 channel (at line 30) because we train the model on one channel images. The image we received from front end has 3 channel (RBG) with the shape (280, 280, 3) and we convert to (280, 280).

We need to resize the image to 28 x 28 (at line 33). We need resize the image because our original size was 280 x 280. You can check on canvas tag with width and height of 280 (at line 43). Remember our model was trained using 28 x 28 image, so if we want to predict any images, we need to resize into 28 x 28. Also expand the array dimension from (28, 28) to (1, 28, 28) because the number 1 means we have 1 images with the size 28 x 28.

STEP #3: Create Drawing App

We are going to use HTML5 canvas to create a simple working drawing app which can be use to write digits using mouse. Then we can send into our server into base64 image data. Create new folder named templates inside the app folder, then create new file named drawing.html

Please take a look at the {% %} and {{ }} tags in the html file. This is the template engine of flask which can be used to add logic for the user views. We use these logic to display the prediction result from backend.

The script tag in the HTML file is a client side javascript that use to make an interaction with canvas by listening to the mouse and that are drawing and erasing. Also the javascript would convert the canvas into base64 image when the user clicking the detect button

STEP #4: Deployment

Now we have our app running well, next step is to deploy the app to heroku. Go to heroku dashboard, create new app with the name as you want. Go to settings, copy the git url and follow these steps.

1. Initialize git in your project directory using git init ,
2. Commit your changes using git add . then git commit -m “Your message”
3. Set the main branch to master using git branch -M master
4. Add heroku remote to your git repository by using git remote add heroku your_heroku__git_app_url

Note: The latest name of master branch is main, but heroku still use master as the name of the main branch. You can name the main branch as you want

Create wsgi.py in main / root project directory

Create Procfile in main / root project directory. Please note that the file has no extension and start with capitalize character. This is what inside Procfile.

web: gunicorn wsgi:app

The Complete Directory Tree

Digit Recognition Application
│
│   Procfile
│   wsgi.py
│
├───app
│   │   ai.h5
│   │   main.py
│   │
│   └───templates
│          drawing.html
│          home.html
│
└───static
        script.js
        style.css

Now you are all set, start testing your app in your local application first by running your wsgi file pthon wsgi.py or by running flask run . The app may start slow on first time, especially because of tensorflow which need quite time. Once it is started you should see something like this.

* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Go to that url in your browser and you should see your app.

Number recognizer application demonstration

After making sure everything is working properly in localhost we can deploy our project to heroku to be accessible by public online. git add . to trace everything in your repository and then made commit by git commit -m "your commit message" . Finally push your code into heroku by using git push heroku master since the master branch is your main branch and the heroku is your heroku remote url. If the process succeed and there is no error shown, this mean your website is successfully deployed.

Photo by bruce mars on Unsplash

Resolution

Congratulations, your website is successfully deployed, go to the website to making sure everything is working before you share it to your colleagues. That’s all, you have fully working drawing app using AI. Please note that you still can improve the application by adding extra features, such as make it real time detection. You can also create a visualization of the prediction probability in front end using visualization libraries.

You can also access my Github repository for material and examples https://github.com/PhilipPurwoko/NumberRecognition

Also read my another articles for more materials and tutorials

How to Make Mask Detection AI Using Python in 6 Steps