Automated Photo Quality Curation
Introduction
The world is quickly moving towards a digital future. People tend to preserve memories of the events in ever growing stream of digital photos and videos. It’s easy to snap a photo then another one and one more from a different angle or just in case it will be a better shot. It’s fun taking lots of photos with friends and family, selfies, occasion and travel shots..
What is not so fun tough is having to dig through a large collection of lost of similar photos with only a few really good ones that deserve being shared, printed or even looked at. Both everyday amateurs as well as professional photographers are faced with the similar challenge of curating the photo collection to reduce the clutter and only keep the beautiful, high quality shots that stand out and delight us.
In the following series of posts I will help you address this challenge with the help of the state-of-the-art technology for automated photo quality curation. We will learn to build and apply tools from the domain of Deep Learning and Machine Vision. Explore the basic building blocks for a product based on a trained neural network model. Experiment with methods based on predefined feature analysis and compare that with a machine learning based solution. We implement and integrate a fully functional tool for automated photo quality curation for Adobe Lightroom. Finally we explore the challenges of taking a working prototype from a lab environment into production.
Topics to cover:
1. Convolutional Neural Network trained to predict photo quality score
2. Filtering out non-photo images
3. Clustering similar images and choosing the best one
4. Face detection, feature extraction
5. Naive version of blink and smile detection
6. Neural Network based blink and smile detection
7. Adobe Lightroom integration
8. From prototype to production
Prerequisites
Before we dive into a magical world of neural networks it’s good to define measurable goals, prepare the development environment and build the testing scaffold which will let us iterate quickly with rewarding feedback cycle.
We would need:
- Software development experience with Python, general curiosity and patience.
- A collection of photos (100+) which will be used to test the system at each of the stages. To automate the process we will first manually label and rate them. This is out testing set which we set aside and only use to verify the results. We never use it in any of the model training activities.
- For each of the functions of the system that will be implemented using a neural network model we would need a representative set of labeled samples for training and automated validation. To get reasonable results you would normally need 10000+ samples however we will be using a smart technique of transfer learning which helps to both reduce the amount of samples required as well as the time it takes to train the model.
- The neural network training process is computationally intensive however it’s possible to use parallel processing to speed it up significantly. GPUs are great at parallel computation of floating point math. Specifically Nvidia CUDA capable GPUs allow to reduce the training times several orders of magnitude.
- Finally we would need a programming and testing environment capable of handling our neural network training tasks and ideally integrated with visualisation and analysis tools.
Setting up the Lab
It used to require a high-end desktop machine or a hefty priced public cloud GPU enabled server to experiment with deep neural networks. Today it’s possible to get a free cloud instance of a development environment with a GPU backend thanks to Googles generous offering of Colaboratory platform.
It would allow us to install the required tools, setup experiments and visualise the results. If you are not familiar with Google Colaboratory nor you have used Jupyter notebooks there is an excellent guide with a detailed overview of key features and common usage scenarios: Google Colab Free GPU Tutorial
In the next post I will cover the end-to-end process of training a deep convolutional neural network to predict the quality rating of the photo.
