What the heck is a callback?!?!?!
What’s a callback?
A callback is a function that you write in case something happens. When that something happens, you call back (hence the name) that function to do something.
This recipe shows a simple way of implementing callbacks in Python. There are a few ways this can be done. The way shown here uses a simple function-based approach.
In a nutshell, a callback can be informally described like this: function a calls function b, and wants to make b run a specific independent chunk of code at some point during b’s execution. We want to be able to vary which chunk of code gets called in different calls to b, so it cannot be hard-coded inside b. So function a passes another function, c, to b, as one argument, and b uses that parameter c to call the functionality that a wants b to call. (Function b may pass some parameters to the function represented by c, when it calls it. These could be either internally generated, passed from a, or a combination of both). So, by changing the value of the function c that gets passed to b (on different calls to b), a can change what chunk of code b calls.
The program callback_demo.py (below) shows a simple way to implement and use callbacks, using just plain functions, i.e. no classes or anything more sophisticated are needed:
The function callback_a (the last argument to the call to processor), gets substituted for the function parameter callback in the processor function definition. So callback_a gets called, and it reports the progress of the work being done. If we passed a different callback function instead of callback_a, we could get different behavior, progress report in a different format, or to a different destination, or something else, without changing the actual processor function. So this is a way of creating functions with low coupling, or putting it in other terms, creating functions (like processor) that have high cohesion.
Callbacks in the training loop
Let’s have a look at the basic fastai / PyTorch training loop first:
There are four main steps, shown in the image:
In order to be able to use functions in between all of those steps, you simply have to add a list callbacksof functions that we will call at every step in case there’s something to do. Something like this:
You can see that now we can modify our training loop pretty much however we want, at every step. We just added callbacks everywhere.
And this is how it looks like after implementing the callbacks:
