Transfer Learning: Massive training data reduction with pre-training
The main concern of companies that want to incorporate into their business any feature or product that makes use of machine learning algorithms is the need of big amounts of labelled data to train such algorithms. For this reason, one of the main goals of MoonVision is to achieve high data efficiency by solving the clients’ needs with just a fraction of the data our competitors ask for. For example, in detection problems, we have successfully delivered projects with just 10 images per object category. One important technique to achieve such data efficiency is the use of pre-trained models and transfer learning.
When approaching a project, we first do research about what data sets are available that could contain object instances related to the problem we want to solve, and then use them to pre-train a model. For example, if our project entails detecting different types of dishes, we will look for a data set that includes multiple food categories.
Once we have pre-trained our model and are happy with the results and the choice of the model architecture, we use transfer learning (also known as fine-tuning) with the client’s data to achieve much better results on the objective task than if we would have trained our model from scratch. In essence, transfer learning allows us to take advantage of the similarities between the client’s data objects and the ones in the data used for pre-training. Then, since the model has already seen similar object instances to the ones we are interested in, it only needs to adjust itself during the new training to accommodate the new object instances in the client’s data.
Even when using a pre-trained model, we still have a lot of flexibility regarding which layers of the pre-trained model we want to keep and any additional pre-processing that we want to apply to the client’s data.
So, what are the advantages of following this approach?
For the client, this means less effort from their side and faster delivery of their orders. Time savings start at the labelling data generation and continue with the creation of the model itself.
But not all is about quick delivery, we use some of this saved time to improve results. Having a pre-trained model on the right data means that we already start with high accuracy, and our job is to improve it further, always aiming at perfect results.
