HOME CREDIT DEFAULT RISK — An End to End ML Case Study — PART 3: Deployment using Flask on AWS EC2

“No machine learning model is valuable, unless it’s deployed to production” — Luigi Patruno

Most of the projects and models built around the world by students, practitioners, and researchers never really make it past their local computers, or rather their Jupyter Notebooks. If there is one most overlooked concept in the field of Machine Learning, it definitely has to be the Deployment and Productionization of the models.

We all are great machine learning modelers. We might know everything from data loading, to cleaning, analysis, feature engineering, modelling, inference, etc., but when it comes to implementing the model in real-time, most of us go blank. We tend to have no clue as to how to put the model into a real-time environment where for the incoming inputs, we would have to keep generating outputs.

But what really is Deployment and Productionization? Well, they are not that hard to understand. Machine Learning is employed to solve real-world problems. Now, what good is a solution to a problem, if it doesn’t meet its end users? This is where these two terms come up. Deploying the model into Production is like taking unused Gym Equipment lying in the warehouse to an actual Gym, where people can use it for a workout. Thus, productionization means taking the Machine Learning model to the consumers, where they can send in inputs and receive the predicted outputs.

Machine Learning Project Workflow (Source)

In this final blog of the series, we will look at how we can deploy the model built in the previous blog into an AWS server, where anyone from anywhere in the world can get the predictions generated from our model, without having to run our personal Jupyter Notebook.

Table of Contents

1. Pre-requisites
2. Building the pipeline
3. Building the Web-API using Flask
4. Setting up AWS Instance and Web-app remotely
5. End Notes
6. References

1. Pre-Requisites

In the previous blog post, we had completed building our final model, i.e. by doing feature engineering, hyper-parameter tuning, making predictions, and calculating metrics. It is now time to use that model for making predictions on incoming data.

Before moving forward, first we’d need to save some files. Which files are we actually talking about? Say, for example, as we saw during feature engineering, we predicted some missing values of features of EXT_SOURCE using an XGBoost Regression model. Now, what if, during the testing time, we come across a data point having a missing value of EXT_SOURCE features? We would need that Regression model to predict the missing values, and for that purpose, we need to save the Regression model that we had trained during the Feature Engineering.

Thus, the task at hand is to look for every variable, model, values, etc. that we had used during training time, and might need for doing transformations to test data to bring it to the required shape and size. We also need to save the final model, that we would need for making the predictions during testing time.

One such great way of saving variables is by using the pickle module.

The pickle module implements binary protocols for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream (from a binary file or bytes-like object) is converted back into an object hierarchy.

— https://docs.python.org/3/library/pickle.html

It is very simple to save a variable to a pickle file, and then load it back to memory, and can be done with just 2 lines of code as shown below.

Code snippet for using pickle to dump and load variables

Once we have identified the variables that we’d need for predictions during test-time, it's time to move to the next step.

2. Building the Pipeline

Now we have to create a pipeline that can streamline the whole process of getting input and generating an output in just one line of code, by making use of classes and functions.

Pipeline (Source)

Kindly note that although the best model that we had obtained in the last blog was an XGBoost model with 600 input features, however, since we are going to be deploying our model in a fairly low-specs server (1 GB RAM, single-core), yeah that’s what you get for free 😛, we will train a LightGBM model which is a lot faster than the former, with just top 300 features. The model training notebook can be found out from my GitHub repo — here.

Now that we have our final sets of features and our final classification model, we can build the final pipeline, as pipeline.py. This python file contains a class named final_pipeline which contains 4 member functions, viz. __init__, load_required_files, preprocessing, and predict. We don’t need to explain what these functions would do, as their names are self-explanatory.

The whole pipeline is a little bit too long for this blog post, so we’ll just see the function calling and output from the Jupyter Notebook.

Loading the sample test data-points and instantiating the object of final_pipeline class

• In the above image, we first load the testing data table, from which we will use a random data point to get the prediction.
• We then create an object of the class final_pipeline.

Making the prediction on a test data point

• In the first block of the above image, first randomly sample a test data point from the test dataset, and display it in the output.
• Next, we call the predict method from our class final_pipeline, which gives us the predicted class probability and the label, where 0 denotes Non-Defaulter and 1 denotes Defaulter. It also returns some features whose values can be used for the interpretation of the predicted class label.

Let us look at what is actually happening inside the predict function.

predict function from final_pipeline class

• From the above code snippet, we observe that this function creates a connection with a Database, which contains the data about the applicant’s previous loans, as we had seen from the EDA portion.
• Next, it calls another function from the same class, i.e. the preprocessing function that preprocesses the data to the required format. Once the pre-processing has been done, our data is in the format exactly required by the model to make predictions.
• We first make a prediction from each of the 3 folds models, that we had trained, and then average the prediction over these folds to get the final predicted probability.
• Now we convert this probability to the class label, based on the optimal threshold found during the training phase, and return the predicted probability, class label along with some important feature values.

So, we saw that our pipeline is working as expected, i.e. giving a sensible output for each input, without any errors. The next step is to somehow bring this input-output framework to a web-based environment, where user can give the input and get the output on a web-page.

3. Building the Web-API using Flask

Flask API (Source)

Flask is a web framework. This means flask provides you with tools, libraries and technologies that allow you to build a web application.

Flask basically helps to build Web-APIs in just a few lines of code. We’ll take a look at the code for better understanding.

Code block for app.py

The above code snippet is enough to build our complete Web-API (yes it is that simple), although we also have to design the HTML pages, with which this API would interact. We’ll try to understand the code line by line.

• Firstly, we are importing a bunch of libraries. We are also importing the final_pipeline class from pipeline.py, which we had seen above.
• Next, we instantiate the Flask object, with a variable name ‘app’, using app = Flask(__name__). Here the value of __name__ will depend on the way this file app.py has been accessed. If we are directly accessing it, it will have a value equal to '__main__'. In the last line of code as well, we check if this python file has been accessed directly or not, and if it has, then we run this Flask based web app on the URL localhost:5000.
• We also create an object of our final_pipeline class named test_predictor_class.
• Now, we open up a helper file, which we had discussed earlier.
• In our web-app, we have three webpages, namely the Home-Page, Prediction page and the Results page.
• If the URL has just ‘/’ in the end, the line @app.route('/', methods = ['GET']) gets executed, which calls the home_page() function. Here methods = ['GET'] indicates that we can only access this function via the URL, and not by some other webpage. This function returns a rendered HTML template named the home-page.html.
• In the next line we have @app.route('/home', methods = ['POST','GET']). This line of code implies that if the URL has /home in it or the action, /home, has been posted by some other web-page, the function inputs_page() gets called. This function renders another HTML template, named predict.html which contains a form that takes in an SK_ID_CURR as input. Here, we have used the methods = ['POST','GET'], which implies that this line can be executed both if some other function calls it, or if it is accessed via the URL.
• In the next line lies our most important function, i.e. @app.route('/predict', methods = ['POST']). This line of code implies that this line has to be executed if some other web-page posts a /predict action. The function below it, i.e. prediction() function is the most useful function in all of this python file. Let us see what this prediction function does.

1. First, we create a connection with the Database.
2. Next, we get the SK_ID_CURR as input from the user.
3. The input received from the web-page can be accessed in JSON format by using request.form. We convert this JSON format to a dictionary and get the value corresponding to the key ‘SK_ID_CURR’.
4. We then type-cast it to integer datatype and fetch the data point corresponding to it from the database.
5. Next, we call the predict function from our final_pipeline’s object test_predictor_class which returns the predicted probability, class label, and the important feature values.
6. The prediction() function finally renders the web-page named result_and_inference.html and passes the required data to be printed to this web-page.
7. This web-page then finally displays the Predicted Results.

Structure of Folders

Structure of the Folders for working with Flask

The Flask API requires the data to be arranged in a particular format, as shown above. All the HTML pages need to go to the templates folder, all the images in static/images and the CSS files in static/stylesheets.

HTML Pages

Now that we have understood some of the most basic and the only required functionalities of Flask, we are ready to deploy our model, but just for the sake of completeness of the blog, let us also have a look at the HTML pages that we have built.

a. home-page.html

home-page.html template

This is the code for the home-page.html file, which contains just one form, which basically has a submit button, which is used to redirect to the predict.html page by interacting with the Flask API.

The method = ‘POST’ and action = ‘/home’ informs flask to call this @app.route('/home', methods = ['POST','GET']) line of code. Everything has started to make sense now, isn’t it?

b. predict.html

predict.html template

This HTML page is used to get the SK_ID_CURR as input from the user, through a form, whose value lies between 100001 and 456255. Note that here we are not getting the actual statistics about an applicant from the user, because there are 120 raw features relating to the stats of an applicant, and it’d be cumbersome to create such a large form just for the sake of practice. Thus, we are only getting the current application ID from the user, and fetching the records of that applicant from the SQL Database.

Again, from the above image, the line action = '/predict' method = 'POST' informs flask to execute the prediction() function.

c. result_and_inference.html

result_and_inference.html template

This HTML page is used to display the final Predictions of the Model, which includes the predicted Class Label, predicted Probability of that Class and the important features’ values.

{% for table in tables %}
    {{ table|safe }}
{% endfor %}

The above block is used for printing the table which we are returning from the predict function of the pipeline, containing the important features’ values.

We also display the predicted class probability and class label using the code:

Based on the model's prediction, the Client is <b><u>{{output_class}}</u></b> with a probability of <b><u>{{output_proba}}</u></b>

Accessing the Web-App

To access this web-app, firstly run the app.py file directly. Now, we just need to go to the URL that we had talked about before, i.e. localhost:5000/. The final 3 webpages would look like as shown in below images.

home-page.htm page

predict.html page

result_and_inference.html page

Phew! That was easy.

Now, we are just left with setting up an instance on AWS and deploying the above-created web-app there. To make it all easy for the readers, I have created a separate folder named Deployment in my Repository, which you can download directly and use.

Note: Due to the file size limitation in GitHub, we could not upload the Database file in the repository itself. The Database can be downloaded from the Google Drive link — here. After downloading this Database, just paste it in the Final Pipeline Files folder.

4. Setting up AWS Instance and Web-app remotely

I would not be able to go through each and every step of setting-up the AWS instance, but the readers can go through this — link, which has a complete step-by-step guide for setting up a Free Tier EC2 instance.

Once we have created the instance, it is just a cakewalk from there.

a. Open Command Prompt/Terminal.

b. Navigate to the folder in which we have saved our key pair.pem file, using CD <directory path>.

c. Access the terminal of the EC2 instance by using the SSH command. Type in the below-mentioned command:

ssh -i "<key pair name>.pem" ubuntu@<Instance's Public IPv4 DNS>

If the above command would have been entered correctly, we’d see a terminal like the one shown below.

Terminal of EC2 Instance

d. Now that we are into the instance’s terminal, we first need to install the Python and its dependencies, which can be installed by the below mentioned commands one by one.

sudo apt-get install python3-pip
pip install <each of the following packages>
Packages needed:
pandas
numpy
pickle-mixin
xgboost
lightgbm
flask

e. Now, we need to copy the whole folder named Deployment, which contains all the files discussed in this post along with the API that we have built, to this instance. To do this, we first open another Command Prompt/Terminal and navigate to the directory where we have both the key pair.pem file and the Deployment folder, that contains the web-API files. Next, we type the below-mentioned command:

scp -r -i "<key pair name>.pem" ./Deployment ubuntu@<Instance's Public IPv4 DNS>:~/

f. If we do it correctly, we see your files getting uploaded to the EC2 instance in the command prompt.

g. Once the files have been copied completely to the EC2 server, we go back to the EC2 instance’s terminal and type ls. We can see the uploaded folder now. We now navigate to that folder using cd Deployment.

h. Now we type the below mentioned command to start our web-app on this server.

nohup python3 app.py &

i. This would run the web-app on the EC2 instance. Now to access it remotely, all we have to do is copy our instance’s Public IPv4 DNS and add:5000 at the end, and boom, we’ve gotten your deployed model.

AWS Instance Properties, to locate Public IPv4 DNS

To try out my deployed model, the readers may visit this link mentioned below.

http://ec2-18-222-96-92.us-east-2.compute.amazonaws.com:5000/

Note: Only the t2.micro instance is eligible for free tier, you’ll be charged for any other instances. Do not forget to stop the instance once you are done with it.

5. End Notes

Just like all good things have to come to an end, this series is no different. With this, we have reached the end of our series of blog posts. In these 3 blogs, we have covered an end to end ML case study starting from scratch to deploying it for end-users. I believe it was quite a long series, but I hope the content makes up for it.

I’d be more than happy to receive critical suggestions and feedback, so kindly feel free to put those in the comments.

For any doubts, or queries, the readers may also comment in the blog post itself, or connect with me on LinkedIn.

The whole project can be found in my GitHub Repository linked below.

rishabhrao1997/Home-Credit-Default-Risk

References

1. pickle — Python object serialization
2. AWS EC2: Create EC2 Instance (Linux)
3. Deploying a machine learning model on the Web using Flask and Python.
4. AppliedAICourse