De-mystifying python decorators

When I first started programming in python I was confused by decorators. I don’t remember where I first encountered them, but it was probably in a framework like flask, and probably looked something like this:

@app.route("/")
def hello():
    return "Hello World!"

This example comes from the flask documentation, and shows how a function is added to the flask application’s routing table. After the module containing the definition above is imported the flask framework will know how to route requests to “/” to the “hello” function. So how does this work?

Let’s simplify that example even further and pick it apart:

@foo
def bar():
    print "In bar"    
    return

One of the best and simplest explanations of this behavior I’ve seen came from a stackoverflow post. As it stated, the code above is equivalent to:

def bar():
    print “In bar”
    return
bar = foo(bar)

We haven’t yet shown the definition of foo() so let’s add that:

def foo(func):
    print "In foo"
    return func
def bar():
    print "In bar"    
    return
bar = foo(bar)

Python functions are objects, just like strings, lists, numbers, everything else in the python type system. In fact python functions are of type ‘callable’. When you define a function like ‘bar’ in the example above the body of the function is the callable, and a reference to that callable is assigned to the name ‘bar’. The next line in the example, ‘bar = foo(bar)’ reassigns the name ‘bar’ to refer to whatever is returned from ‘foo(bar)’. For obvious reasons that needs to be a callable, which it is in this case since ‘foo’ just returns the callable it was passed.

Now let’s take that completed example and put it back in decorator form, stick it in a module, and run it:

# deco_test.py
def foo(func):
    print "In foo"
    return func
@foo
def bar():
    print "In bar"    
    return
if __name__ == "__main__":
    print "Starting up"
    bar()
$ python deco_test.py
In foo
Starting up
In bar
$

If that’s what you expected to see, then you’ve got it. You can probably stop reading. If not, then carry on.

The key to understanding the output from this example is understanding when the various parts of the code run. Let’s return to our “undecorated” example for a moment:

def foo(func):
    print "In foo"
    return func
def bar():
    print "In bar"    
    return
bar = foo(bar)

Recall that the @decorator syntax is just shorthand for reassigning the name of ‘bar’ to refer to a callable returned from ‘decorator’, which in this case is ‘foo’. When does ‘foo’ run? It runs when it is called. Since it is called on the right hand side of the assignment to ‘bar’ it will run when that assignment is performed. Since that assignment is module level (not inside a class or function definition) it will be performed when the module is imported, like any other module level code. Decoration, then, always happens at import time.

That makes sense, especially in the context of something like the flask framework example that led off this article. The routing to get http requests to the handler function needs to be set up once, and module import is a fine time to do that. But what about things like authentication? Can’t decorators be used to “wrap” a function and add new behavior at call time rather than import time? They can, and this capability relies on the fact that python supports nested or “inner” function definitions:

# deco_test.py
def foo(func):
    f = func
    print "In foo"
    def wrap_func():
        print "In wrap_func"
        return f()
    return wrap_func
@foo
def bar():
    print "In bar"    
    return
if __name__ == "__main__":
    print "Starting up"
    bar()
$ python deco_test.py
In foo
Starting up
In wrap_func
In bar
$

In this example the decorator runs at import time, stores the passed callable ‘func’ in a local variable, and then returns the inner callable ‘wrap_func’. The result of this is that the name ‘bar’ now refers to ‘wrap_func’ and the original callable that ‘bar’ referred to is stored in the local. When ‘bar()’ is called later it is actually ‘wrap_func’ that runs and gets a chance to do some work, such as authenticating a token, before passing control on to ‘bar’. The reason that this works at all is due to another important python topic called “closures,” but that is a subject for a future post. For now it’s enough to say that a closure preserves the state of the outer function’s arguments and local variables as they were at the point where the inner function’s definition was encountered. That’s why the local variable ‘f’ is available for ‘wrap_func’ to refer to well after ‘foo’ has already run and returned.