Advanced Python made easy — 2

Source

In the previous article, we’ve gone through several useful features of python programming language in general. So, consider this just a continuation of previous article where we’ll extend it with some additional concepts obviously using decorators i.e without disturbing the actual content of previous article.

Decorators

The concept of decorator presents one of the most beautiful and powerful design possibilities not only in the domain of python but also in whole realm of software design. Essentially decorators are just wrappers, modifying the behavior of code mostly by extending the functionality without needing to change what it is wrapping. To make the concept crystal clear, let’s start with building foundation.

Functions aka First class objects

A function simply returns a value based on the given arguments. In python, these functions are given an additional honor of first-class objects. This honor was bestowed upon functions appropriately considering they can be passed around and used as arguments just like plain objects. For example, they can be passed as arguments to other functions as well as used as return value of function.

Functions as arguments

def greet(name):
    print ('Hello ' + name)
def send_greetings(fun, name):
    fun(name)
send_greetings(greet, 'John')

Inner Functions

Inner functions are defined inside some other function. As a result, the inner functions are not defined until the parent function is called OR they are locally scoped to parent OR they only exist inside parent as local variables.

def send_greetings(name):
    def greet_message():
        return ‘Hello ‘
    result = greet_message() + name
    print (result)
 
send_greetings(‘John’)

Returning Functions from Functions

Python also allows to use function as a return value for some other function. Essentially, we just return the reference to the inner function which we can call later.

def classify(element):
    def even_number():
        print ('Element is even.')
    def odd_number():
        print ('Element is odd.')
    if element%2 == 0:
        return even_number
    else:
        return odd_number
classify(2)()

Decorators

Now, with all these basic concepts under our belt, let’s fit the pieces together to form a complete picture.

def my_decorator(fun):
    def wrapper():
        print (‘Before calling the function…’)
        fun()
        print (‘After calling the function…’)
    return wrapper
def say_hello():
    print (‘Hello!’)
say_hello = my_decorator(say_hello)

That’s it!!!. Here is the simplest decorator we can ever have. We literally just applied what we learned so far. So a decorator is

A function that takes another function as an argument, generates a new function, augmenting the functionality of the original function, and finally returning the generated function so we can use it anywhere.

Additionally, python makes it bit cleaner and nicer for programmers to create and use decorators.

def my_decorator(fun):
    def wrapper():
        print (‘Before calling the function…’)
        fun()
        print (‘After calling the function…’)
    return wrapper
@my_decorator
def say_hello():
    print (‘Hello!’)

Context Managers

Simply put, a context manager is a resource acquisition and release mechanism that prevents resource leaks and ensures proper cleanup even in the face of deadly exceptions. For e.g ensuring file getting closed after opened, lock getting released after acquired. This concept is clearly expressed and properly utilized in dozens of other programming languages as well such as RAII in C++.

Technically, it’s a simple protocol that an object needs to follow. This protocol requires, for an object to behave as context manager, to implement __enter__ and __exit__ methods.

__enter__ returns the resource to be managed whereas __exit__ does any cleanup work and does not return anything.

class File:
    def __init__(self, name):
        self.name = name
    
    def __enter__(self):
        self.file = open(self.name, 'w')
        return self.file
    def __exit__(self, type, value, trace_back):
        if self.file:
            self.close()

Now the above class can be used safely within with block. More generally, using with, we can call anything that returns a context manager.

with File('example.txt') as f:
    f.write('Hey hello')
    f.write('See you later. Bye!!!')

__enter__ is called when execution enters the context of with statement. When execution leaves with block, __exit__ is called.

Context managers could be used for more complex problems as well. Let’s see another example where the need for context managers is nigh unavoidable. The resource in question is lock and the problem we could avoid is mighty deadlock.

from threading import Lock
lock = Lock()
def do_something():
    lock.acquire()
    raise Exception('Oops I am sorry. I have to raise it!')
    lock.release()
try:
    do_something()
except:
    print ('Got an exception.')

Notice, that exception is raised before releasing the lock. The obvious after effect of this is all other threads calling do_something will be stuck forever causing system to be dead-locked. Using context manager we can get rid of this nasty situation.

from threading import Lock
lock = Lock()
def do_something():
    with lock:
        raise Exception('Oops I am sorry. I have to raise it!')
try:
    do_something()
except:
    print ('Got an exception.')

Whoa! See even in the face of certain exceptions, cleanup will be done properly. Clearly there’s no reasonable way to acquire lock using context manager and end up not releasing it. That’s the way how it should be.

Chained Exceptions

Consider a situation where a method throws ZeroDivisionError because of an attempt to divide a number by Zero. Clearly, we have good set of tools of handle exceptions in python. However, what if cause of this exception is TypeError in some other function called by this same function. In that case, it would not be enough to just report the top level exception as we will loose the origin and hence the main cause of chain of exceptions. To demonstrate the concept in question, let’s take a simple example.

def chained_exceptions():
    try:
        raise ValueError(17)
    except Exception as ex:
        raise ValueError(23) from ex

if __name__ == "__main__":
    chained_exceptions()

In python 2.0, situations like above would lead to latter exception being reported whereas former being lost, as depicted below.

Traceback (most recent call last):
File “test.py”, line 3, in chained_exceptions
    raise ValueError(23)
ValueError: 23

Clearly, we have lost a valuable piece of information since we lost the actual cause of exception i.e it’s origin. However, running the same script with python 3.0, we would get complete exception stack trace.

Traceback (most recent call last):
File “test.py”, line 3, in chained_exceptions
    raise ValueError(17)
ValueError: 17
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “test.py”, line 8, in <module> chained_exceptions()
File “test.py”, line 5, in chained_exceptions
    raise ValueError(23) from ex
ValueError: 23

Please let me know through your comments any modifications/improvements needed in the article.