So Call Me…Maybe?

Callbacks for deep learning (part 1)

No…this article isn’t about your normal “call” like the famous Carly Rae Jepsen song! It’s about “callbacks’ in coding! As we’ll later see in part 1 — and in the future part 2) — callbacks can be a very important programming concept with useful applications for deep learning models. Gear up, because this is definitely going to sound ‘crazzzzzy’!

What are Callbacks?

Let’s begin by visualizing an example of how callbacks operate. GUIs, or graphical user interfaces, are a great example. Specifically, GUI widgets are items/icons we encounter almost on a daily basis — take your favorite social media website or even Google search, for example, when you click on a widget (e.g. ‘Search’ or ‘Login’) you usually expect it to perform a certain action, or event. However, widgets in coding by themselves do not have a pre-defined and set function, rather the operation we expect to happen must be passed into it as a function, or, in more technical terms, we pass in an object.

So, next, when we click Search to browse a certain topic or Login to open up our social media profile page, what essentially that is going on in the backend of our operation is a callback! Let’s take a visual example with some bit of code to understand this.

In Jupyter Notebook, (credit to Jeremy Howard), we create a simple graphical widget, Click Me. Clicking on the created widget itself doesn’t have any response, but what about when we assign the function f(o) to it? f(o) is defined to print the phrase “Hi”. Next, f(o) is passed into our Click Me widget as an object pointed to it by our widget; ergo the next time we decide to click Click Me, it callsback on f(o) to perform the event, print “Hi” (see figure 1)! Sounding crazier yet?

Fig. 1: Printing “Hi” When Clicking the Widget

Other examples where we’ve seen callbacks in action? Well, when you have a GUI widget displaying various plot combinations based on certain parameters passed into it, that’s a real-time situation of implementing callbacks. You can choose to adjust the parameters inputted for plotting function, allowing you to create various graphical patterns and understand unique insights for each. View this cool example below (see figure 2) — you’ve probably played around with something similar before!

Figure 2: Callbacks for Other GUI Widgets, e.g. Number of Flight Arrival/Departure Delays by Airport & Airline

And In Deep Learning…?

In deep learning, callbacks can become very useful for executing operations, and are technically referred to as Hooks in this context. But before we get to that in part 2, let’s take the time to simply break down the various callback operations which exist. Can an event have more than one callback operation? Are they created similar to defining a function with an input argument? If so, how many arguments can they take? Let’s find out!

Fig 3: Creating a callback to show the progress for an event, calculating time.

To get started, here’s an example of a callback function, Slow Calculator (see Figure 3), created in FastAI we’re going to use to visualize the forms of these operation types:

1. Lambda Notation

If you’re already familiar with Python, Lamba expressions are a simple and direct way to create a new function and perform an operation, usually with a single line of code. Essentially, we can define a function at the point we want to perform an execution without having to necessarily define and create a new one. For our purposes with callbacks, we can define one at the same time we want to use it for an event. Here’s how our callback looks in lambda notation taking in a single parameter for Slow Calculator (see Figure 4):

Fig. 4: Lambda Notation for Showing Progress for Slow Calculator function.

2. Partials & Two Argument Callbacks

Fig. 5: Callback function make_show_progress with a 2nd argument, exclamation

What if want to include a particular exclamation with our show progress callback function? Our callback takes in one argument, so we need to find a new way to include two arguments for a function that only takes one parameter. So the way we do this is by defining a function within a callback and carrying a value from an external context (e.g. the exclamation) into the callback function and storing it there (see Figure 5). For Javascript programmers, this is called, Closure. Fortunately for us, Python already has a tool able to this for us, called a Partial Function Application. Overall, Partial (see Figure 6) helps with taking a function that takes two parameters and turning it into a function that takes one parameter.

Fig. 6: Using the Python tool Partial to take a function with two parameters and turn it into one taking only one

3. Callbacks As A Callable Class: Multiple callback functions *args and **kwargs

Fig. 7: Implementing a callback using a class

So what do we mean by this? Exactly as it sounds! You may also recognize that this extending upon the concept of Closure recently discussed, however where we originally used a function to build a closure for our defined state we could similarly use a class to build a callback (see Figure 7). Specifically, continuing on our operation of adding an exclamation to a time calculation for a function, we could directly store the state (exclamation) inside self and pass it into the initializing (__init__) function. Moreover, __call__ is a special function that is called if an object is called using parentheses (in our case Cb), printing our stored state together with a value from an external context. Thus, when we call our time calculation function, Slow Calculation, it does exactly as we expect, displaying the epoch, or run time, as the passed in external value.

*args and **kwargs are also an interesting topic for us to dive into, but right now, for our purposes in this article, we will at least summarize the usage of *args and **kwargs for callbacks (see Figure 8):

• *args takes in positional arguments and returns a tuple called args.
• **kwargs takes in keyword arguments returning a dictionary kwargs.
• One common use case for these is for wrapping another class or object, ergo we may take some arguments as *args and **kwargs and pass it on to another functional object. However, need to be careful in implementation because items quite often not in the parameter list of a function may end up in *args and **kwargs. When we usually want to quickly throw things together, *args and **kwargs can be particularly helpful for this.

Fig. 8: Displaying the function that takes in arguments *args and **kwargs and its return values

So What’s Next?

In part 2, we’ll dive deeper into Hooks, or callbacks in deep learning when building neural networks, and look at the ways FastAI creates and implements it. Stay tuned — It’s going to get to a whole new level of crazy!