Python — Understanding Advanced Concepts with Ease: Day 7 (Itertools, Functools and Magic Methods)

Photo by David Clode on Unsplash

Welcome back to Day 7 of our Python — Understanding Advanced Concepts with Ease series! Let’s learn something new today.

Itertools

Python’s itertools module is a treasure trove of powerful tools for working with iterators and iterables. It is a part of Python's standard library, making it readily available for use in your projects without any additional installations.

It offers a wide range of functions for performing common iteration-related tasks, such as permutations, combinations, and Cartesian products, among others.

Photo by Dmitriy Demidov on Unsplash

Key Functions in itertools

Infinite Iterators:

• itertools.count(start=0, step=1): Generates an infinite arithmetic progression.
• itertools.cycle(iterable): Repeats elements of an iterable indefinitely.
• itertools.repeat(element, times=None): Repeats an element indefinitely or a specified number of times.

Combinatoric Iterators:

• itertools.permutations(iterable, r=None): Generates all permutations of elements in an iterable.
• itertools.combinations(iterable, r): Generates all combinations of elements in an iterable.
• itertools.product(*iterables, repeat=1): Generates the Cartesian product of input iterables.

Terminating Iterators:

• itertools.islice(iterable, stop): Returns an iterator that generates items from the input iterable up to the specified stop index.
• itertools.takewhile(predicate, iterable): Returns elements from the iterable as long as the predicate is true.
• itertools.dropwhile(predicate, iterable): Skips elements from the iterable until the predicate becomes false, then returns the remaining elements.

Grouping and Partitioning:

• itertools.groupby(iterable, key=None): Groups consecutive elements of an iterable by a common key function.
• itertools.partition(pred, iterable): Partitions the elements of an iterable into true and false elements based on the predicate function.

Examples

import itertools

colors = ['red', 'green', 'blue']
sizes = ['S', 'M', 'L']

cartesian_product = list(itertools.product(colors, sizes))
print(cartesian_product)

Output:

[(‘red’, ‘S’), (‘red’, ‘M’), (‘red’, ‘L’), (‘green’, ‘S’), (‘green’, ‘M’), (‘green’, ‘L’), (‘blue’, ‘S’), (‘blue’, ‘M’), (‘blue’, ‘L’)]

functools

Python’s functools module is a powerful utility for working with higher-order functions, decorators, and other functional programming constructs. It provides a set of tools to manipulate and extend the behavior of functions, making it easier to work with them in various scenarios.

Let me explain with some examples:

Partial Functions

functools.partial() allows you to create partial functions from existing functions by fixing a certain number of arguments. This is useful when you want to create a new function with some arguments already specified.

from functools import partial

# Define a function
def power(base, exponent):
    return base ** exponent

# Create a partial function
square = partial(power, exponent=2)

# Now `square` is a function that squares its argument
print(square(5))  # Output: 25

Function Decorators

functools.wraps() is a decorator used to preserve the metadata of the original function when creating decorators. It copies attributes such as __name__, __doc__, and __module__ from the original function to the decorator function.

from functools import wraps

def my_decorator(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print("Before function execution")
        result = func(*args, **kwargs)
        print("After function execution")
        return result
    return wrapper

@my_decorator
def my_function():
    print("Inside the function")

my_function()

Function Composition

Although not directly provided by functools, function composition can be achieved using reduce() from the built-in functools module in combination with lambda functions.

from functools import reduce

def compose(*funcs):
    return reduce(lambda f, g: lambda x: f(g(x)), funcs)

add_one = lambda x: x + 1
multiply_by_two = lambda x: x * 2

composed_function = compose(multiply_by_two, add_one)
print(composed_function(3))  # Output: 8 (2*(3+1))

Memoization

functools.lru_cache() is a decorator that caches the results of a function, thus saving computation time when the function is called with the same arguments again.

from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    if n <= 1:
        return n
    return fib(n-1) + fib(n-2)

print(fib(10))  # Output: 55

Magic methods

In Python, magic methods are functions whose name begin and end with two underscores, like __ init__(), __add__(), __len__() or __repr__().

These are also known as dunder methods (short for double underscore)

If you run help(obj) for an object, you can see the magic methods for that object, like help(int) or help(str).

For example, help(int) shows me —

When you run an object method (or built in function) in a program (for example for a len), it calls its corresponding magic method internally.

Why are these called magic methods?

Photo by Aziz Acharki on Unsplash

Magic methods behave differently depending on the data types involved in the operation. The behavior of a magic method is determined by (1) the objects on which the method is called and (2) the context in which it is used.

For example, consider the __add__() method, which defines the behavior for addition (+). When you use the + operator with two numbers, the __add__() method of the left operand is called, and the right operand is passed as an argument. However, when you use + with two strings, it performs concatenation, which is defined by the __add__() method of strings.

Similarly, the behavior of comparison methods (__eq__(), __lt__(), __gt__(), etc.) depends on the data types being compared. For numeric types, these methods implement standard numerical comparison logic. For sequences like lists or tuples, these methods compare elements pairwise. For custom classes, you can define these methods to implement comparison logic specific to your class's semantics.

The behavior of each magic method is determined by the specific implementation provided by the developer for a given data type. Hence, these are called magic methods.

Conclusion

Congratulations! Hope you learnt something new today. See you on Day 8.

Happy coding!