Double the ROI of your Machine Learning Investment with Transfer Learning
NEWS ALERT: Google’s AlphaGo team trains computer to …
beat a game you’ve never heard of.
Yes, researchers are making machine learning (ML) breakthroughs which have incredible potential.
Exciting research? Absolutely.
But most of these research advances are still … research.
Those of us solving today’s business problems with ML rely on off-the-shelf ML algorithms trained through supervised learning.
Unfortunately getting real results with these proven solutions has many challenges. For instance, supervised learning requires labeled training data — data which can be expensive.
In this post I’ll discuss an approach for reducing this cost through a technique called transfer learning.
Exciting as Google’s latest research? Nope.
Just practical.
Supervised learning 101
Supervised learning is a technique for training prediction algorithms with labeled training data.
For example, suppose you want to predict the calorie count of a meal from a picture. You could start with a set of training examples of meals labeled with calorie count data.
A data scientist could use these labeled examples to train an algorithm to predict the calorie count of a dish from a picture.
Normally a ML algorithm requires ~5K labeled training examples to achieve any results. At least 50K labeled examples are required to achieve human-level predictive capabilities.
Sounds easy enough right? Well … probably not.
Getting labeled data is the ML project budget-buster
The big hurdle in supervised learning is getting a sufficiently large corpus of labeled training data. Using our example above, estimating calorie count for each image would require information like a list of ingredients, cooking method and weight for every dish.
How can you get your data scientist enough data?
Let’s assume your data scientist asks for 100k pictures of food labeled with an accurate calorie count. How could you get her this much data?
- Hire 100 chefs to generate meals, record ingredients, and take pictures of them?
- Crowdsource it to chefs worldwide through some sort of promotional incentive?
Getting thousands of labeled training examples can take a lot of time, money — or both.
Transfer learning to the rescue
Overcoming these data challenges is a major area of research.
One solution is transfer learning, a technique for taking knowledge gained while solving one problem and applying it to a related problem.
A data scientist first identifies free or inexpensive labeled datasets and trains algorithms with them. She then trains the same algorithms with a smaller set of labeled data to make the predictions.
Transfer learning applied to our food example
Let’s go back to predicting the calories in a meal from its picture.
Suppose your CFO only approves enough budget to generate 1,000 pictures of meals labeled with calories — a mere 1% of what your data scientist requested. Before begging for more $$ you try generating results through transfer learning.
- Start with free public data. Begin training the algorithms using a subset of the 14 million labeled examples in the ImageNet dataset. This process begins training the algorithms to recognize image features.
- Generate training data with Amazon’s Mechanical Turk. Take 50,000 food images and hire mechanical turks to label the visible ingredients. Then train the ML algorithms on them. This step will teach the algorithm how to identify features such as the plate, parts of food, edges, etc.
3. Train using your limited labeled data. Train the ML algorithms using the 1,000 labeled images with calorie counts.
What the ML algorithms learn in the first 2 steps is “transferred” to improve learning in the final step to make the desired predictions.
Predicting poverty in African villages with satellite imagery using transfer learning
My fictional example of predicting meal calories is similar to a successful technique by Stefano Erman of Stanford. In Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping, Erman demonstrates how transfer learning can be used to train algorithms for estimating poverty in African countries.
Getting accurate poverty data from poor, rural areas is a notoriously hard problem. Doing on-the-ground human surveys is expensive and often dangerous.
Using just a tiny amount of labeled survey data Erman was able to predict poverty levels in rural villages. He started training his algorithms with free ImageNet and satellite data. He then transferred this learning to make accurate poverty predictions using a small number of on-the-ground surveys.
The results are quite amazing.
Transfer learning is a practical technique for solving real business problems
In the real world we never have enough data.
Unless you’re Google or Facebook, getting labeled data can be prohibitively expensive. Transfer learning techniques provide 2 primary business benefits:
Transfer learning means faster experiments
A key component of Lean Startup innovation is testing ideas by running fast experiments.
Have an idea for a new service offering based on your organization’s data? Before you spending millions on a multi-year effort first try generating early success using transfer learning.
Transfer learning means higher ROI from ML projects
ML data requires an ongoing investment as environments change. Transfer learning can reduce the cost of ongoing data managements and boost the ROI of any ML project.
What happens when you combine machine learning, Lean Startup, IOT, blockchain and AI? I have no idea. But if you want to talk about using machine learning in your business please contact me at kevindewalt@kevindewalt.com.
