What is Transfer Learning? — Idiot Developer
Transfer Learning is a technique in machine learning where we reuse a pre-trained model to solve a different but related problem. It is one of the popular methods to train the deep neural network. It is generally used for image classification tasks where the amount of the dataset is small.
In this article, we will go through what transfer learning is, how it works and the advantages it offers. Additionally, we will also cover the most common problems related to it.
Transfer learning is a research problem in deep learning where a model trained on a task is reused to initialize another model for training a similar task.
Table of Contents:
- What is transfer learning?
- How does transfer learning work?
- Why should we use transfer learning?
- Further reading
- Summary
What is Transfer Learning?
Human beings have the ability to utilize the knowledge learned across different tasks. We are able to transfer the knowledge learned from a task to another similar task easily.
For example:
- If you know how to ride a bike, you can easily learn to ride a car.
- If you know mathematics, you can easily learn physics.
In conventional deep learning, models are only trained to learn only one type of task. To learn another task, we have to build the model again from scratch. To overcome this limitation, we will learn the concept of transfer learning.
Transfer learning is a research problem in deep learning where a model trained on a task is reused to initialize another model for training a similar task. For example, you have an image classification model trained for wild animal classification. You can use this pre-trained model and use it for dog breed classification. It is mostly used in computer vision and natural language processing tasks.
READ MORE:
In terms of deep neural networks, we use the weight of the pre-trained model and use it to initialize another deep neural network to create a new model. This new model contains the features learned by the previous pre-trained model. In this way, we exploit the knowledge gained from a task to improve the performance in another task
How does transfer learning work?
In a deep convolutional neural network (CNN) we have different layers which learn different features from data. These features ultimately form the entire deep neural network. These features detect lines and edges in the earlier layers and shapes in the middle layers. The features in the last layers are task-specific. In transfer learning, we use the features from the earlier and the middle layers and train the last layer from the scratch. This helps us to take the advantage of the features learned by the pre-trained model on the previous task.
The earlier and middle layers consist of general features which can easily be used for other similar tasks. While the last layers or classifier layers consist of task-specific features. These features need to be learned from the new dataset.
Why should we use transfer learning?
Transfer learning offers several advantages over the existing approaches.
- Save training time.
- Require less labelled data.
- Better performance.
- Better generalization capability.
A deep neural network consists of a lot of parameters that require a lot of data. The training process takes a lot of time, sometimes it takes days or even weeks when the dataset is really huge. With the use of the transfer learning approach, we don’t train the model from scratch, so it requires less labelled data and takes less training time.
As, we are using a pre-trained model, which is already trained on a wide variety of data. So, the newly formed model would have better performance even in the case of an extreme scenario. This would give the model a better generalization capability and allow it to be more robust.
Further Reading
- A Gentle Introduction to Transfer Learning for Deep Learning
- Transfer Learning — CS231n Convolutional Neural Network for Visual Recognition
- A Comprehensive Hands-on Guide to Transfer Learning with Real-World Applications in Deep Learning
- What Is Transfer Learning? Exploring the Popular Deep Learning Approach
- A Survey on Transfer Learning — Research Paper
- How transferable are features in deep neural networks? — Research Paper
Summary
In this article, you have learned what is transfer learning and how it is better than the classical deep learning approach. Specifically, you have learned:
- What is transfer learning?
- How does transfer learning work?
- Why should we use transfer learning?
Still, have some questions or queries? Just comment below. For more updates. Follow me.
Originally published at https://idiotdeveloper.com on August 24, 2021.
