Understand the decorator pattern and its implementation in Python.
In Python, functions are first class citizens. This allows us to assign function to variables, pass functions through arguments and return function from another function.
Example:
# Accepting function as argument
def logger(operation: callable) -> None:
print(f"Operation {operation.__name__} started")
operation()
print(f"Operation {operation.__name__} ended")
logger(lambda: print("Hello World!"))
# Output:
# Operation <lambda> started
# Hello World!
# Operation <lambda> ended
# Return function from function
def counter() -> callable:
count: int = 0
def increment() -> None:
nonlocal count
count += 1
print(f"Counter: {count}")
return increment
increment_counter = counter()
increment_counter()
increment_counter()
increment_counter()
increment_counter()
# Output:
# Counter: 1
# Counter: 2
# Counter: 3
# Counter: 4Closures uses this first class function feature and allows us to return function (from another function) which remembers local variables of the scope in which it was created, even when its in some other scope.
In the above example, counter function returns increment function, and later when we call the increment_counter, the function still remembers the count variable of counter function’s scope. This is closure.
Anatomy
Decorator is a design pattern which allow us to add new behavior to functions/objects dynamically without altering the structure (code) of them. Decorators can be created using functions (mostly preferred) or classes.
A function decorator is a function that takes another function as argument, then adds functionality to that function, and then returns another function.
This power extending and modifying existing function/object without altering the source code is both powerful and leaky, and should be used carefully.
The following example displays the functionality of a decorator:
def decorator_funciton(original_function: callable) -> callable:
def wrapper_function() -> None:
print(f"Clean up resources before running: {original_function.__name__}")
return original_function()
return wrapper_function
def compuation():
print("Computation is running")
decorated_compuation = decorator_funciton(compuation)
decorated_compuation()
# Output:
# Clean up resources before running: compuation
# Computation is runningHere, we’re able to add functionality to computation without altering the function itself. Python has built-in support for decorators, so instead of running the decorator function, and then running the returned function, we can decorate computation function with decorator_function. Both are functionally same.
def decorator_funciton(original_function: callable) -> callable:
def wrapper_function() -> None:
print(f"Clean up resources before running: {original_function.__name__}")
return original_function()
return wrapper_function
@decorator_funciton
def compuation():
print("Computation is running")
compuation()
# Output:
# Clean up resources before running: compuation
# Computation is runningIn case our function receives arguments, we’ve to pass these argument to the wrapper function. This can be done using args and kwargs.
In the following example, we’re passing arguments to the compuation function:
def decorator_funciton(original_function: callable) -> callable:
def wrapper_function(*args, **kwargs) -> None:
print(f"Clean up resources before running: {original_function.__name__}")
print("Arguments: ", args)
print("Keyword Arguments: ", kwargs)
return original_function(*args, **kwargs)
return wrapper_function
@decorator_funciton
def compuation(ram: int, cpu: int, storage: dict):
print("Computation is running")
print(f"RAM: {ram} GB")
print(f"CPU: {cpu} Cores")
print(f"Storage: {storage}")
compuation(4, 2, {"ssd": 256, "hdd": 1024})
# Output:
# Clean up resources before running: compuation
# Arguments: (4, 2, {'ssd': 256, 'hdd': 1024})
# Keyword Arguments: {}
# Computation is running
# RAM: 4 GB
# CPU: 2 Cores
# Storage: {'ssd': 256, 'hdd': 1024}Class Decorators
We can also use classes to create decorators. During initializtion (i.e. in __init__ method) we defined the original function (for later use) and the __call__ method becomes the wrapper function.
class DecoratorClass:
def __init__(self, original_function: callable) -> None:
self.original_function = original_function
def __call__(self, *args, **kwargs) -> None:
print(f"Clean up resources before running: {self.original_function.__name__}")
print("Arguments: ", args)
print("Keyword Arguments: ", kwargs)
return self.original_function(*args, **kwargs)
@DecoratorClass
def compuation(ram: int, cpu: int, storage: dict):
print("Computation is running")
print(f"RAM: {ram} GB")
print(f"CPU: {cpu} Cores")
print(f"Storage: {storage}")
compuation(4, 2, {"ssd": 256, "hdd": 1024})
# Output:
# Clean up resources before running: compuation
# Arguments: (4, 2, {'ssd': 256, 'hdd': 1024})
# Keyword Arguments: {}
# Computation is running
# RAM: 4 GB
# CPU: 2 Cores
# Storage: {'ssd': 256, 'hdd': 1024}Without the decorator annotation (@) we’ll have something like this:
decorated_function = DecoratorClass(compuation) # __init__
decorated_function(4, 2, {"ssd": 256, "hdd": 1024}) # __call__Sometimes class decorators are more easy to implement as compared to function decorators. Function decorators are mostly used.
Usage
While decorators are powerful, it has some drawbacks. Unnecessary or too much use of it’ll increase complexity of our codebase. It hides details from the user and it make code harder to understand and debug. Just like other design pattern, it should be used based on a problem.
Here we’ll go through few examples of using decorators to solve problems.
1. Logger
Decorators can add logging functionality to functions, allowing us to log inputs, outputs, and other relevant information.
from typing import Any
# =========================
# Function decorator
# =========================
def logger(func: callable) -> callable:
def wrapper(*args, **kwargs):
output = func(*args, **kwargs)
print(f"Function '{func.__name__}' called with args: {args}, kwargs: {kwargs}")
return output
return wrapper
@logger
def addition(x: int, y: int) -> int:
return x + y
print(addition(3, 5))
# Output:
# Function 'addition' called with args: (3, 5), kwargs: {}
# 8
# =========================
# Class decorator
# =========================
class Logger:
def __init__(self, cls: "Addition") -> None:
self.cls = cls
def __call__(self, *args: Any, **kwds: Any) -> Any:
instance = self.cls(*args, **kwds)
print(
f"Class '{self.cls.__name__}' instantiated with args: {args}, kwargs: {kwds}"
)
return instance
@Logger
class Addition:
def __init__(self, x: int, y: int) -> None:
self.x = x
self.y = y
def __call__(self) -> int:
return self.x + self.y
print(Addition(3, 5)())
# Output:
# Class 'Addition' instantiated with args: (3, 5), kwargs: {}2. Authentication
Using decorator to check for authentication:
from typing import Any
# =========================
# Function decorator
# =========================
def authenticate(func: callable) -> callable:
def wrapper(*args, **kwargs):
print("Authenticating user...")
return func(*args, **kwargs)
return wrapper
@authenticate
def upload_image() -> int:
print("Uploading image...")
upload_image()
# Output:
# Authenticating user...
# Uploading image...
# =========================
# Class decorator
# =========================
class Authenticate:
def __init__(self, func: callable) -> None:
self.func = func
def __call__(self, *args, **kwargs) -> Any:
print("Authenticating user...")
return self.func(*args, **kwargs)
@Authenticate
def upload_image_v2() -> int:
print("Uploading image [v2]...")
upload_image_v2()
# Output:
# Authenticating user...
# Uploading image [v2]...3. Taking arguments
Decorators can also take arguments. Let’s say we want to retry some functionality for n number of times:
from typing import Any
# =========================
# Function decorator
# =========================
def retry(num_retries: int) -> callable:
def decorator(func: callable) -> callable:
def wrapper(*args, **kwargs) -> Any:
for _ in range(num_retries):
try:
return func(*args, **kwargs)
except Exception as e:
print(f"Retrying... {e}")
return wrapper
return decorator
@retry(num_retries=3)
def upload_image() -> int:
raise Exception("Failed to upload image")
upload_image()
# Output:
# Retrying... Failed to upload image
# Retrying... Failed to upload image
# Retrying... Failed to upload image
# =========================
# Class decorator
# =========================
class Retry:
def __init__(self, num_retries: int) -> None:
self.num_retries = num_retries
def __call__(self, func: callable) -> callable:
def wrapper(*args, **kwargs) -> Any:
for _ in range(self.num_retries):
try:
return func(*args, **kwargs)
except Exception as e:
print(f"Retrying... {e}")
return wrapper
@Retry(num_retries=3)
def upload_image_v2() -> int:
raise Exception("Failed to upload image [v2]")
upload_image_v2()
# Output:
# Retrying... Failed to upload image [v2]
# Retrying... Failed to upload image [v2]
# Retrying... Failed to upload image [v2]Notice that in case of using function decorator, we now have 3 nested functions where the outer most function i.e. retry take input arguments.
4. Multiple decorators
We can also attach multiple decorators to functions/objects.
from typing import Any
# =========================
# Function decorator
# =========================
def uppercase(func: callable) -> callable:
def wrapper(*args, **kwargs) -> str:
result = func(*args, **kwargs)
return result.upper()
return wrapper
def exclaim(func: callable) -> callable:
def wrapper(*args, **kwargs) -> str:
result = func(*args, **kwargs)
return f"{result}!"
return wrapper
@uppercase
@exclaim
def greet(name: str) -> str:
return f"Hello {name}"
print(greet("John"))
# Output:
# 'HELLO JOHN!'
# =========================
# Class decorator
# =========================
class UpperCase:
def __init__(self, func: callable) -> None:
self.func = func
def __call__(self, *args, **kwargs) -> str:
result = self.func(*args, **kwargs)
return result.upper()
class Exclaim:
def __init__(self, func: callable) -> None:
self.func = func
def __call__(self, *args, **kwargs) -> str:
result = self.func(*args, **kwargs)
return f"{result}!"
@UpperCase
@Exclaim
class Greeter:
def greet(self, name: str) -> str:
return f"Hello {name}"
print(Greeter().greet("John"))
# Output:
# 'HELLO JOHN!'In the above example we’re using wraps decorator from functools to preserve the metadata of the decorated function. If we don’t do that then the function passed in the next decorator wrapper looses the original function metadata (name, etc…).
The order of execution is from top to bottom:
def decorator_1(func: callable) -> callable:
print("Decorator 1", func.__name__)
def wrapper_1(*args, **kwargs):
print("Wrapper 1", func.__name__)
return func(*args, **kwargs)
return wrapper_1
def decorator_2(func: callable) -> callable:
print("Decorator 2", func.__name__)
def wrapper_2(*args, **kwargs):
print("Wrapper 2", func.__name__)
return func(*args, **kwargs)
return wrapper_2
@decorator_1
@decorator_2
def my_function():
print("My function")
my_function()
# decorator_2(decorator_1(my_function))()
# Output:
# Decorator 2 my_function
# Decorator 1 wrapper_2
# Wrapper 1 wrapper_2
# Wrapper 2 my_function
# My functionIn the above output, wrapper_1 doesn’t have info of my_function like wrapper_2 does. We can preserve this metadata using meta decorator from the functools package.
from functools import wraps
def decorator_1(func: callable) -> callable:
print("Decorator 1", func.__name__)
@wraps(func)
def wrapper_1(*args, **kwargs):
print("Wrapper 1", func.__name__)
return func(*args, **kwargs)
return wrapper_1
def decorator_2(func: callable) -> callable:
print("Decorator 2", func.__name__)
@wraps(func)
def wrapper_2(*args, **kwargs):
print("Wrapper 2", func.__name__)
return func(*args, **kwargs)
return wrapper_2
@decorator_1
@decorator_2
def my_function():
print("My function")
my_function()
# decorator_2(decorator_1(my_function))()
# Output:
# Decorator 2 my_function
# Decorator 1 my_function
# Wrapper 1 my_function
# Wrapper 2 my_function
# My function