Intel Image Classification with PyTorch (Pt2): Transfer Learning with Pre-trained Ensemble Model
I. INTRODUCTION
In deep-learning, obtaining training data and the process of training itself are costly, both in terms of physical effort and computational resources, with no guaranty of excellent performance. In my previous article (below), a VGG-like model was trained from scratch on the Intel Image dataset and could only hit about 70% test accuracy. In reality, many applications used transfer learning instead of training the models from scratch to take advantage of the pre-trained weights for many complex architectures trained on vastly larger datasets.
The idea behind transfer learning is that there are underlying high-level features that are common across many use cases. These are features learnt in earlier layers of a deep learning model, such as edges and blobs. Only the low-level features that are learnt in much deeper layers will provide the final semantically differentiating value in a specific use case. Architecturally, these 2 parts are coined the “body” and “head”…