1. Business Problem
1.1 Problem Description
WhatsApp has over 1.2 billion daily active users today. WhatsApp has become an indispensable part of our lives. The app has made it fast and easier to stay connected with our friends. However, it can be a little annoying at times when the chat app automatically downloads all those stupid memes, audio files, videos and photos. This not just makes the phone’s gallery look like a complete mess, it also eats up your mobile memory. Too many WhatsApp images, videos making your phone slow down. Deleting the whole WhatsApp Images folder is a bad option because it may contain our important personal images. So we need to somehow separate out unnecessary memes picture and our personal images.
Can we build a model that can identify memes images and real images?
What you can learn from the below blog?
- How to fine-tune pre-trained Keras models. (Transfer Learning)
- Deploy a flask app on Google Cloud servers.
- Deploy a MobileNet model on an android platform using TF-Lite library.
1.2 Problem Statement :
Classify Memes images from the phone gallery without touching personal images. Find out memes images from the gallery.
1.3 Real-world/Business Objectives and Constraints :
- The cost of misclassification can be very high. Because suppose if an image is not meme and we classified it is as meme then it not good for app.
- Latency concerns. The model should quickly identify image memes or not. Otherwise, app threw into the trash.
1.4 Machine Learning formulation
It is simply a two-class classification problem. We need to build a classifier that can discriminate memes images and not memes images.
1.5 Performance Metric:
We will consider binary cross-entropy or LogLoss as a performance metric. We will consider Accuracy as a performance measure.
Build an android app name as MemesDetectionApp, which can differentiate “meme” images and “Not a meme” images. So that we can separate out memes image from phone gallery and delete all unnecessary images.
2.1 Data Acquire
Frankly speaking there is not readymade dataset available for the type of real-world problem. This dataset we created by gathering images from various data sources.Some data we collected from the below link.
2.2 Dataset size
We collected almost 11k memes image and 11k not memes images from the internet. We ensure that t dataset is balanced.
2.3 Data Cleaning
There are so many images extensions available in the market. Something like JPG, JPEG, PNG, SVG, GIF, PSD, RAW and so on.
We decide to focus on JPG and JPEG images. We try to convert all images to JPG format.
2.4 Sample Not-Memes Images/ Real Images
We are going to train two different models.
- One model which can identify “Meme” and “Not Meme” image locally within android environment.
- Another model will be deployed somewhere on the server and we can make HTTP REST API request to a server and the server will respond in JSON format, whether a requested image is “Meme” or “Not Meme”.
We tried VGG-16, VGG-19, ResNet-50, Inception model.
Below is the Keras pre-trained INCEPTION model graph. We can see that highly overfitting model. It is not learning anything, so we will discard this option.
Below is the RESNET50 model. It is quite better than the inception model, but yet not better than VGG19.
After trying various model we decided to use VGG19 model.
We’ve performed hyperparameter tuning on below params,
- added or removed layers after Top layer.
- tried different dropout rates
- choose different optimizers
- tried different learning rates
- tried different batch size
After experiments on various values, we find optimal values those are as follows,
- No additional layer after TOP. (ie TOP= false)
- Dropout == 0.5
- Optimizer == Adamax
- Learning rate == 0.0005
- Batch size == 100
We have used TensorBoard extensively for hyperparameter tuning.
Training LogLoss = 0.19593
Training Accuracy = 97.95%
Validation LogLoss = 0.28459
Validation Accuracy = 97.20154%
Test LogLoss = 0.25857
Test Accuracy = 97.43742%
After visualizing Training loss vs Validation loss graph and above result, We can surely say that a model is NOT OVERFITTING.
Then we can save the fine-tuned model for further operations.
model3.save("/content/drive/My Drive/Meme Detection/best_model.h5")
The crucial part of this case study project is deploying a model on Google cloud servers and Android platform.
5. Android APP
1. Landing Page
- Check single Images Locally
- Check Multiple Images Locally
- Check Single Image on Server(Flask REST API)
- Check Multiple Image on Server(Flask REST API)
1. Check single Images Locally
It means Mobile-Net tflite model is running locally and testing only a single image.
2. Check Multiple Images Locally
It means Mobile-Net tflite model is running locally and we are testing multiple images.
3. Check Single Image on Server(Flask REST API)
It means fine-tuned VGG19 model running with flask app on GCP server and testing only a single image.
4. Check Multiple Image on Server(Flask REST API)
It means fine-tuned VGG19 model is running with a flask app on GCP server and testing on multiple images.
- There was not readymade dataset available for this case study we create one, collecting various images from the internet. Dataset is balanced.
- We chose binary cross-entropy as a loss metric and accuracy as a performance metric.
- Tried various models like VGG16, VGG19, ResNet50 and Inception model. Out of which VGG19 was better performing.
- We then fine-tuned the model, with various hyperparameters.
- Finally, we achieved 97.44 % Test accuracy and log loss reduce to 0.258
Don't miss deploying tf-lite model to the android platform part2.
Also, don't miss deploying Keras model to the flask app part3
Please appreciate our efforts if you like this blog.
Github source code can be found here.
Special Thanks to