To get or not getattr, with or without attr

From wikimedia

Some days ago Adrià Fontcuberta asked me if that piece of code (that he read in a code review) was ok:

url = biography.image.url if biography.image else ''

In fact, this is a common statement when using ORMs (this case was in a Django application).

My first answer was:

Well, it’s not a real mess, but in Python is common the use of EAFP (Easier to ask for forgiveness than permission) coding style.

The point is the use of try / except statements instead of if statements. The Python docs say that this style is clean and fast and can avoid race conditions between the looking and the leaping of the attribute in a multi-threaded environment.

I also showed him another way to do the same, with getattr, delegating to Python what do you want and what to fallback as a default, and getting a clean, readable (but reflective) statement:

url = getattr(biography.image, 'url', '')

And here could have finished. But during the weekend my brain recovered the conversation and thought:

Ok Angel, are you sure that all of this is true? In any case? Considering the cost of handling an exception? Considering the cost of reflection?

So I made a little experiment.

The experiment

So let’s write two minimal classes in order to make the experiment:

class Foo:
    def __init__(self, bar=None):
        self.bar = bar
class Bar:
    def __init__(self, baz):
        self.baz = baz
foo_with_bar = Foo(Bar(42))
foo_without_bar = Foo()

And then, let’s write some functions to test different ways of getting the baz value if bar is set:

def test_if(foo):
    baz = None
    if foo.bar:
        baz = foo.bar.baz
    return baz
def test_ternary(foo):
    return foo.bar.baz if foo.bar else None
def test_exception(foo):
    try:
        baz = foo.bar.baz
    except AttributeError:
        baz = None
    return baz
def test_getattr(foo):
    return getattr(foo.bar, 'baz', None)

And let’s use IPython and his timeit magic command to check the performance of every case:

WITH BAR SET
%timeit test_if(foo_with_bar)
174 ns ± 0.195 ns per loop (...)
%timeit test_ternary(foo_with_bar)
166 ns ± 0.495 ns per loop (...)
%timeit test_exception(foo_with_bar)
154 ns ± 0.113 ns per loop (...)
%timeit test_getattr(foo_with_bar)
180 ns ± 0.141 ns per loop (...)

WITHOUT BAR SET
%timeit test_if(foo_without_bar)
129 ns ± 0.162 ns per loop (...)
%timeit test_ternary(foo_without_bar)
132 ns ± 0.106 ns per loop (...)
%timeit test_exception(foo_without_bar)
497 ns ± 7.69 ns per loop (...)
%timeit test_getattr(foo_without_bar)
174 ns ± 0.859 ns per loop (...)

For readability reasons, I replaced the following text with (...):
(mean ± std. dev. of 7 runs, 10000000 loops each)

So, let’s analyze the results:

• The difference between a full if condition and a ternary if condition is insignificant (about 2%–5%). This is logical considering that they’re two different syntactic ways of doing the same thing.
• The getattr approach is slower than the if approach, reducing an insignificant ~3% the performance if bar is set and a considerable~35% if is not set.
• Using the EAFP exception handling approach when bar is set and the exception is never raised improves significantly the performance (about 12% better than the if approach).
• But using the EAFP exception handling approach when bar is always None and the exception is always raised, reduces the performance a 285%.

Conclusions

• Leaving aside the recommended EAFP coding style of Python, avoid using exception handling for flow control. Handling exceptions have a performance cost (and it’s notable when the exception is not exceptional) and using them for flow control can lead to a misunderstanding of the intent of the code.
  It is not an unbreakable rule, there are occasions where using this style can help you to have your clean, fast and thread-safe approach, but in most use cases you will downgrade the performance of your application trying to (prematurely) optimize it.
• Although getattr can make your code more readable and avoid writing twice the variable like in the if or ternary approaches, reflection has a cost in terms of performance and robustness (for example, if you’re using the typing module it will return always Any and you will lose some of the benefits of the static type checker).

And the most important conclusion: probably, you don’t have to look at this as a performance problem. The applications that have this kind of ORM traversing statements normally have other real performance problems and the nanoseconds that you can win analyzing this kind of statements are ridiculous.

So my advice here is:

• Unless you know what are you doing and what performance problem are you solving, don’t use the try / except approach and avoid premature optimizations of this kind.
• If you and your team are ok with some reflection on the code, don’t be afraid of using getattr and just ignore the performance difference.
• If you’re concerned about typing and want to start the journey of the static type hinting in Python, avoid reflection methods and use one of the conditional approaches.