What is Self-Supervised Learning? | Will machines ever be able to learn like humans?
Introduction
In deep learning, you should always aim to start your neural network training with a pre-trained model, and fine-tune it. You really don’t want to be starting with random weights, because it means that you’re starting with a model that doesn’t know how to do anything at all. With pretraining, you can use 1000 times fewer data than starting from scratch. However, sometimes a pre-trained model that is appropriate for your case doesn’t exist. So we would like something which works better, or at least has similar results, but doesn’t need the huge amount of annotated data pre-trained models use. The secret for this purpose is “self-supervised learning”.
This is where we train a model using labels that are naturally part of the input data, rather than requiring separate external labels.
What is Self-Supervised learning?
Self-supervised learning is a relatively recent learning technique where the training data is autonomously…
