Memes Detection Android App using Deep Learning

Pranay Sawant
Jan 1 · 6 min read

1. Business Problem

1.1 Problem Description

WhatsApp has over 1.2 billion daily active users today. WhatsApp has become an indispensable part of our lives. The app has made it fast and easier to stay connected with our friends. However, it can be a little annoying at times when the chat app automatically downloads all those stupid memes, audio files, videos and photos. This not just makes the phone’s gallery look like a complete mess, it also eats up your mobile memory. Too many WhatsApp images, videos making your phone slow down. Deleting the whole WhatsApp Images folder is a bad option because it may contain our important personal images. So we need to somehow separate out unnecessary memes picture and our personal images.

Can we build a model that can identify memes images and real images?

What you can learn from the below blog?

  • How to fine-tune pre-trained Keras models. (Transfer Learning)
  • Deploy a flask app on Google Cloud servers.
  • Deploy a MobileNet model on an android platform using TF-Lite library.

1.2 Problem Statement :

Classify Memes images from the phone gallery without touching personal images. Find out memes images from the gallery.

1.3 Real-world/Business Objectives and Constraints :

  • The cost of misclassification can be very high. Because suppose if an image is not meme and we classified it is as meme then it not good for app.
  • Latency concerns. The model should quickly identify image memes or not. Otherwise, app threw into the trash.

1.4 Machine Learning formulation

It is simply a two-class classification problem. We need to build a classifier that can discriminate memes images and not memes images.

1.5 Performance Metric:

We will consider binary cross-entropy or LogLoss as a performance metric. We will consider Accuracy as a performance measure.

End Goal

Build an android app name as MemesDetectionApp, which can differentiate “meme” images and “Not a meme” images. So that we can separate out memes image from phone gallery and delete all unnecessary images.

2. Data

2.1 Data Acquire

Frankly speaking there is not readymade dataset available for the type of real-world problem. This dataset we created by gathering images from various data sources.Some data we collected from the below link.

2.2 Dataset size

We collected almost 11k memes image and 11k not memes images from the internet. We ensure that t dataset is balanced.

2.3 Data Cleaning

There are so many images extensions available in the market. Something like JPG, JPEG, PNG, SVG, GIF, PSD, RAW and so on.

We decide to focus on JPG and JPEG images. We try to convert all images to JPG format.

Sample memes images
Sample Not Memes images

3. Strategy

We are going to train two different models.

  • One model which can identify “Meme” and “Not Meme” image locally within android environment.
  • Another model will be deployed somewhere on the server and we can make HTTP REST API request to a server and the server will respond in JSON format, whether a requested image is “Meme” or “Not Meme”.

4. Model

We tried VGG-16, VGG-19, ResNet-50, Inception model.

Below is the Keras pre-trained INCEPTION model graph. We can see that highly overfitting model. It is not learning anything, so we will discard this option.

Inception model graph

Below is the RESNET50 model. It is quite better than the inception model, but yet not better than VGG19.

ResNet50 model performance

After trying various model we decided to use VGG19 model.

We’ve performed hyperparameter tuning on below params,

  • added or removed layers after Top layer.
  • tried different dropout rates
  • choose different optimizers
  • tried different learning rates
  • tried different batch size

After experiments on various values, we find optimal values those are as follows,

  • No additional layer after TOP. (ie TOP= false)
  • Dropout == 0.5
  • Optimizer == Adamax
  • Learning rate == 0.0005
  • Batch size == 100

We have used TensorBoard extensively for hyperparameter tuning.

Tensorboard graphs
fine tuned model weights visualization
final model graph for Training loss vs Validation loss.

4. Result

Training LogLoss = 0.19593
Training Accuracy = 97.95%

Validation LogLoss = 0.28459
Validation Accuracy = 97.20154%

Test LogLoss = 0.25857
Test Accuracy = 97.43742%

After visualizing Training loss vs Validation loss graph and above result, We can surely say that a model is NOT OVERFITTING.

Then we can save the fine-tuned model for further operations.

The crucial part of this case study project is deploying a model on Google cloud servers and Android platform.

5. Android APP

1. Landing Page

landing page
  • Check single Images Locally
  • Check Multiple Images Locally
  • Check Single Image on Server(Flask REST API)
  • Check Multiple Image on Server(Flask REST API)
  • Ping

1. Check single Images Locally

It means Mobile-Net tflite model is running locally and testing only a single image.

2. Check Multiple Images Locally

It means Mobile-Net tflite model is running locally and we are testing multiple images.

3. Check Single Image on Server(Flask REST API)

It means fine-tuned VGG19 model running with flask app on GCP server and testing only a single image.

4. Check Multiple Image on Server(Flask REST API)

It means fine-tuned VGG19 model is running with a flask app on GCP server and testing on multiple images.

6. Summary

  • There was not readymade dataset available for this case study we create one, collecting various images from the internet. Dataset is balanced.
  • We chose binary cross-entropy as a loss metric and accuracy as a performance metric.
  • Tried various models like VGG16, VGG19, ResNet50 and Inception model. Out of which VGG19 was better performing.
  • We then fine-tuned the model, with various hyperparameters.
  • Finally, we achieved 97.44 % Test accuracy and log loss reduce to 0.258

Don't miss deploying tf-lite model to the android platform part2.

Also, don't miss deploying Keras model to the flask app part3

Please appreciate our efforts if you like this blog.

You can check out the similar interesting blog here and here.

Github source code can be found here.

Reference:

Special Thanks to

Data Driven Investor

from confusion to clarity not insanity

Pranay Sawant

Written by

I am the one who wants to deep dive into the AI (Artificial Intelligence ) ocean.

Data Driven Investor

from confusion to clarity not insanity

More From Medium

More from Data Driven Investor

More from Data Driven Investor

Why I Fired My Only Freelance Client

More from Data Driven Investor

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade