Member-only story

The Unofficial PyTorch Optimization Loop Song

A fun jingle to help you remember the steps in a PyTorch training loop

Daniel Bourke
TDS Archive

--

extravagant man in all white linen singing in front a computer monitor
Every tutorial needs a song.

I’ve been learning PyTorch.

And if you’re learning PyTorch too but have come from frameworks such as Scikit-Learn or TensorFlow, writing your own training and testing loops may be a foreign concept (both of these frameworks have a beautiful model.fit()system).

To help myself learn I wrote a jingle.

So I’d never forget.

You can see the video version of me singing this song on YouTube.

Torchy, hit the music!

What is an optimization loop?

Before we get to the lyrics, let’s define what I mean by optimization loop (or training loop).

In PyTorch, or machine learning in general an optimization loop steps through a dataset with the goal of training a model (often a neural network) to learn patterns in that dataset.

--

--

Responses (1)