Meta-Learning Is All You Need

Update: This post is part of a blog series on Meta-Learning that I’m working on. Check out part 1, part 2, and part 3.

Neural networks have been highly influential in the past decades in the machine learning community, thanks to the rise of computing power, the abundance of unstructured data, and the advancement of algorithmic solutions. However, it is still a long way for researchers to completely use neural networks in real-world settings where the data is scarce, and requirements for model accuracy/speed are critical.

Meta-learning, also known as learning how to learn, has recently emerged as a potential learning paradigm that can absorb information from one task and generalize that information to unseen tasks proficiently. During this quarantine time, I started watching lectures on Stanford’s CS 330 class on Deep Multi-Task and Meta-Learning taught by the brilliant Chelsea Finn. As a courtesy of her talks, this blog post attempts to answer these key questions:

1. Why do we need meta-learning?
2. How does the math of meta-learning work?
3. What are the different approaches to design a meta-learning algorithm?

Diagram of Model-Agnostic Meta-Learning — Chelsea Finn in “Learning to Learn”

Note: The content of this post is primarily based on CS330’s lecture one on problem definitions, lecture two on supervised and black-box meta-learning, lecture three…