Forget About Catastrophic Forgetting: Google’s Continual HyperTransformer Enables Efficient Continual Few-Shot Learning
Continual few-shot learning techniques enable AI models to learn from a continuous stream of tasks described by a small set of samples without forgetting their previously learned information. This learning paradigm is beneficial in real-world applications such as industrial robotics, where a deployed agent must learn in a dynamic environment with limited observations, and in privacy preservation, where sequential training shares only the model weights without exposing the data.
A Google Research team advances this research direction in the new paper Continual Few-Shot Learning Using HyperTransformers, proposing Continual HyperTransformer (CHT), a model that modifies the recently published HyperTransformer (HT, Zhmoginov et al., 2022) to sequentially update the weights of a convolutional neural network (CNN) based on the information in a new task without forgetting the knowledge learned from previous tasks.
The paper outlines the main advantages of the proposed CHT approach as follows:
- CHT is able to generate and update the…
