Towards Meta RL

A machine learning model generally requires a large number of samples to train.

Humans, on the other hand, are able to learn skills and concepts a lot more efficiently. For example, a kid who has seen cats and dogs only a number of times is able to quickly tell them apart.

Even if you’ve seen a single sample you can recognize the same machine even if pixel by pixel it’s completely different.

People who know how to play tennis can learn how to play ping pong fairly easily. Meta-learning puts forward the idea that machine models could potentially be designed to have similar capabilities — like learning with just a few training samples.

Meta-Learning addresses two key challenges with building machine-learning models today:

1. Sample efficiency: To recognize handwritten digits, we typically read 6000 samples per digit
2. Poor transferability: Machine learning models aren’t able to learn from previous experiences or knowledge.

A good meta-learning model is capable of well adapting or generalizing to new tasks that haven’t been seen before. Meta-learning is also known as model learning to learn.

the meta-learning problem

An optimal meta-learning model is trained over a variety of learning tasks and optimized for the best performance across a number of these tasks. Each task is associated with dataset D which contains feature vectors and true labels.

In this case, the optimal model parameters would be:

It’s the same as a normal learning task except that one dataset is used as one data sample.

Meta-supervised learning:

The way to go about learning or training such a learning procedure (f can be viewed as a learning procedure) is to have data that is in the form of different datasets. Each of these datasets has a number of different input and output pairs.