Let’s teach a machine to recognize facial expressions using CNN
Hola Everyone,
I hope you are enjoying Camping With Python blog posts. So in today’s blog post, I will show you how I went about making a CNN that can predict your facial expressions. I will also share the codes for building and using CNN. I will be using the Tensorflow(Keras) framework. So let’s get started.
I am Anuj Trehan, an undergrad studying in India who is fond of Deep Learning and cricket.
Prerequisites to follow along:
As this is not a guide on how deep learning works, thus there are some pre-requisites for this blog:
- Knowledge of Python and Tensorflow(Keras). You can learn it here.
- Knowledge of Machine Learning(Learning rates, optimizers), Deep Learning particularly CNN and ANNs. You can learn it here.
Steps we will take:
- Getting Data and Preparing it.
- Creating a Model and training it.
- Testing the trained model on images and live feed.
Getting Data and Data Preprocessing
The data for this project is taken from Kaggle. You can get it here. The dataset contains 2 files train.csv and test.csv.
We will be using train.csv for model training it has 27,473 samples which have pixel values (for 48X48 grayscale image each) and a label having value as 0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral. The data distribution looks like following
As you can see that the distribution is not uniform. The majority of faces are of the happy, neutral, sad category. So we will apply data augmentation techniques.
Now we will define the code for loading and preprocessing data.
Building CNN Model
CNN stands for Convolutional Neural Networks. Since their advent, they have acquired the Computer Vision world. They have performed better than humans in many tasks like image classification, object detection, object tracking, Image segmentation. You name it and you will find one or the architecture built for the application. I will not go into the theory much in this post but I will link some resources for beginners so that they can understand the theory behind these. So let’s get to the model architecture.
I will be using Keras a high-level API in TensorFlow which makes deep learning model building and training easy. Below is the code and its explanation.
I trained the model on Google Colab which is an online Jupyter Notebook and offers free GPU access for 12 hrs every day. You can see it here. To use Colab you need to have a Gmail account.
The model can be viewed as follows.
Post Training
After training the model, I got an accuracy of around 67%, which is average performance. I tried with other architectures as well but they also got around the same or less accuracy score. But when I tested it on some Images It performed quite well. You can see the image below for results.
As you can see the model did well on all but one image (2nd one) which should have been classified as disgust but classified as angry. The code to reproduce is given below. I used imutils and OpenCV package to make the montage.
Recognizing on Live Feed
After this, I tried to test the model on the live feed from a webcam. And the results were quite good. You can have a look at the script which I used to predict on my face’s live feed.
I used OpenCV to get the live feed from the camera.
Congratulations on completing the project
I hope this tutorial will help you a lot in your future projects. Share this article with your colleagues and friends.
Thanks for being here with me till the end. I am a beginner in writing blog posts so please do comment below about the blog, your comments mean a lot to bring the best content to you.
You can follow me on Twitter, LinkedIn, Facebook
If you find yourself with any doubts or want to share your valuable suggestion do comment below
The next article will be up soon until then keep practicing.
Thank You!!
