Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks ( MAML )

Human can learn a skill within a short time, but in artificial intelligence, agent can’t often do that. This can result from common sense and prior knowledge that human own.

For example, if a agent can stand, then it might learn to walk faster than agent that doesn’t know how to stand. Because many sports need people to stand, with ability of standing, agents might also learn other sports faster.

This kind of problem can be categorized as a meta-learning problem because we want the agent to learn how to learn better.

In paper referenced, they propose a meta-learning algorithm which is general and model-agnostic. This algorithm can be directly applied to any learning problem and model that is trained with gradient descent approach. This method can learn the parameters of any standard model so that it can achieve fast adaptation. The intuition of the method is that some internal representations are more transferrable than others. That is, there are some internal features that are applicable to many of tasks. Changing these parameters can make rapid improvements in tasks we concern. Thus, finding the model parameters that are sensitive to changes in task performance is the goal.

screenshot from paper referenced, diagram of MAML algorithm. The goal is to optimize representation θ that can rapidly adapt to new tasks.

screenshot from paper referenced, T means task and L means loss function, f is a parametrized function with parameter θ.

In the algorithm above, it uses one gradient descent update for simplicity of notation, but actually in the paper, they use one or more gradient descent updates on task Ti.

screenshot from paper referenced, meta-optimization step in the algorithm

The meta-optimization step is performed via stochastic gradient descent (SGD).

The algorithm can be used in supervised learning and reinforcement learning problems, but they are a little different in the form of loss function and how data is generated and presented to the model.

1. For few-shot supervised learning:

screenshot from paper referenced, MAML for few-shot supervised learning

screenshot from paper referenced, Equation (2)

Equation (2) is mean-squared error (MSE), which is a common loss function for regression tasks.

screenshot from paper referenced, Equation (3)

Equation (3) is cross-entropy loss, which is a common loss function for discrete classification tasks.

2. For reinforcement learning:

screenshot from paper referenced, MAML for reinforcement learning

screenshot from paper referenced, Equation (4), R means reward

Reference:

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks