🐣 From zero to research — An introduction to Meta-learning

Thomas Wolf
Apr 3, 2018 · 12 min read
Image for post
Image for post

What’s learning in the first place?

Image for post
Image for post
Single step of the training process of a neural network. The neural net is trained to classify an image as representing a dog or a cat
Image for post
Image for post
Image for post
Image for post
3 steps of a neural net training process where the neural net (large △s) is trained to classify dogs/cats images.

Let’s turn to meta-learning

Image for post
Image for post
Image for post
Image for post
A meta-training step (training the optimizer O) comprising with 3 steps of training the model M)
Image for post
Image for post
The input of a meta-training step is a list of examples (🐈, 🐕) with associated labels (🔺,🔻)

Meta-learning in PyTorch 🔥

Meta-learning in Natural Language Processing 🗣

A meta-learner optimizing a neural net model behaves similarly to a recurrent neural network.

Image for post
Image for post
Image for post
Image for post
The curves indicate how good the model is at predicting the words of a Wikipedia article given the beginning (A, …, H are successive Wikipedia articles), colored words indicate the same for single words, blue is better, red is worse. As the model reads through an article, it learns from the beginning and become better at predicting the end (for more details see our paper).

HuggingFace

Stories @ Hugging Face

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store