What is continual Learning?
The definition of artificial intelligence is the simulation of human intelligence in machines that are programmed to think and act like humans [1]. But are we really simulating human intelligence?
Human Learning Vs Machine Learning
Examples of Human Learning:
- When a person moves to a new city, he adapts to the new environment.
- Also, we learn a new programming language much faster than the first one.
- We incrementally learn new things without forgetting previous knowledge
Machine Learning:
- Supervised learning: learn on a fixed dataset with a fixed number of classes
- Reinforcement learning: learn in stationary, self-contained environments
- Gradient-based optimization assumes that the dataset is balanced (i.i.d.)
- Machine learning leak of adaptation to new enviormnt and tasks
Motivation for Continual Learning
The world is a constantly changing and non-stationary place. For instance, a machine learning model that classifies a dog and a Cat can be asked to classify between a dog, a cat, and torture. In such dynamic environments, Continual Learning becomes crucial. It involves learning from a sequence of tasks, one after the other, without having access to all tasks simultaneously, like in multi-task learning.
Desiderata of Continual Learning
Continual learning involves balancing objectives:
- Minimal access to previous tasks: as the memory is limited, we cannot stock all previous data in the memory
- Minimal increase in model capacity and computation: we cannot add a model whenever a new task is demanded
- Fast adaptation and recovery: If a robot changes its environment, it must adapt to the new environment
- Minimizing catastrophic forgetting (CF): Training on new tasks should not significantly reduce the performance of previously learned tasks. ( I will be writing an article about catastrophic forgetting soon but the idea is that in machine learning, when we learn a new task, we forget the previous tasks )
- Maintaining plasticity: The model should keep learning whenever a new task is observed.
Approaches to Continual Learning
The Continual Learning approaches can be divided into three categories:
- Regularization Techniques: Methods like Elastic Weight Consolidation (EWC) and Synaptic Intelligence introduce regularization terms to protect important parameters learned during previous tasks.
- Replay Methods: These approaches store a subset of previous data to periodically retrain the model on both old and new samples.
- Parameter isolation methods: is divided into Dynamic networks which can grow or adapt their architecture based on the complexity of new tasks, allowing the model to allocate more resources for crucial information and Fixed networks.
Conclusion:
Continual learning is the missing piece in modern machine learning pipelines.
In the next few articles, we will be discussing continual learning approaches in detail, metrics, and benchmarks.
Reference
[1] https://www.investopedia.com/terms/a/artificial-intelligence-ai.asp