Lesser-known Python Tips for Pythonistas!

18 Tips for being a better Python developer: Metaclasses, getters and setters in Pythonic way and more

Hi everyone! Long time, no write! In this article, I’ll be sharing some lesser-known Python tips that I took note while my readings.

Image generated by AI

#1 Method Resolution Order (MRO)

In Python, MRO (Method Resolution Order) is the sequence Python follows to search for a method when it’s called. It starts with the current class and moves up through the parent classes. This is especially important in multiple inheritance, where a class inherits from more than one parent class, as it decides which parent’s method or attribute to use.

Python’s MRO is determined by the C3 linearization algorithm. This algorithm creates an ordered list of all the ancestors (or parent classes) of a class. This list helps Python figure out where to look for a method or attribute in a class’s inheritance hierarchy.

#2 Keys of a Dictionary

Any object that can be hashed can be used as a key in a dictionary.

A hashable object is one that is immutable. This means that sets, dictionaries, and lists, which are mutable, cannot be used as dictionary keys.

So, what makes an object immutable? An object is considered hashable if:

• It has a __hash__() method that returns a unique and constant value (an integer).
• It has a __eq__() method that can compare it with other objects.
• It is immutable, meaning its content cannot be changed after it is created.

#3 LEGB Scope

When the Python interpreter tries to find a name (like a variable or function), it follows the LEGB rule to resolve it. This rule checks different scopes in the following order:

1. Local Scope: The area inside a function or lambda expression.
2. Enclosing Scope: If there’s a nested function within another function (like an outer function), the enclosing scope refers to the area inside the outer function.
3. Global Scope: Names in this scope are accessible throughout the entire Python script (one file with a .py extension).
4. Built-in Scope: This scope includes names that are automatically available when you run a Python shell or script, like keywords (e.g., in, and, or, def, class) and special variables (e.g., __main__, __file__).

#4 `nonlocal` and `global` Keywords

global

• A variable variable is declared in the global scope and initialized with the value "hello 1".
• Inside the fun() function, the global keyword is used to modify this global variable. Without global, Python would treat variable inside the function as a local variable.
• After calling fun(), the global variable is updated to "hello 2", and the change is reflected when printing variable.

variable = "hello 1"  # This variable is declared in the Global scope

def fun():
    global variable  # Without this, we would not be able to change the variable value
    variable = "hello 2"
    
fun()
print(variable)  # Output: "hello 2"

nonlocal (Enclosing scope)

• In the outer() function, variable is declared as a local variable with the value "hello 1".
• Inside the nested inner() function, the nonlocal keyword is used to modify the variable in the enclosing scope (i.e., in outer()'s local scope).
• After calling inner(), the variable in the outer() function is updated to "hello 2", and this change is reflected when printing variable.

def outer():
    variable = "var 1"  # declared in a Local scope
    
    def inner():
        nonlocal variable  # we could not change the variable value without the keyword
        variable = "var 2"
    
    inner()
    print(variable)
    
outer()  # Output: "var 2"

#5 GIL, Global Interpreter Lock

The Global Interpreter Lock (GIL) is a mechanism in Python that ensures only one thread can execute Python code at a time. This lock is managed by the Python interpreter to prevent multiple threads from running Python bytecode simultaneously, making CPython thread-safe. However, it can limit the performance gains of multithreading in CPU-bound programs.

• Multithreading works well for I/O-bound tasks (like network requests or file operations) because the GIL is released while waiting for I/O operations to complete.
• Multiprocessing is often used for CPU-bound tasks to bypass the GIL, as each process has its own Python interpreter and memory space.

#6 Closures

A closure is a special kind of inner function in Python that remembers the variables from its enclosing scope even after that scope has finished executing.

Closures are ideal when you need to maintain state across multiple function calls, or when you want to create lightweight objects that behave like simple classes without using global variables.

import math

def multiply():
    l = []
    def closure(num):
        l.append(num)
        return math.prod(l)
    return closure

current_mult = multiply()
print(current_mult(2))  # Output: 2
print(current_mult(3))  # Output: 6
print(current_mult(4))  # Output: 24

In this example:

• multiply() is an outer function that initializes an empty list l.
• closure(num) is the inner function that adds num to the list and returns the product of all numbers in the list.
• Even after multiply() has finished running, the closure function retains access to the list l, allowing it to remember previous numbers added.

How Closures Work in Python

Closures occur when:

1. You have a nested (inner) function.
2. The inner function references variables from its enclosing (outer) function.
3. The outer function returns the inner function.

The inner function keeps access to the variables from the outer function, even after the outer function has completed its execution.

def outer_function(msg):
    def inner_function():
        print(msg)  # 'msg' is captured from the enclosing scope
    return inner_function

closure_example = outer_function("Hello, Closure!")
closure_example()  # Output: Hello, Closure!

In this example:

• outer_function takes a parameter msg and defines inner_function that prints msg.
• inner_function remembers msg even after outer_function has finished, which makes it a closure.

Why Are Closures Useful?

Closures are useful for:

• Data Encapsulation: They allow you to keep some data private and accessible only through the inner function.
• Factory Functions: They enable you to create functions with pre-configured parameters.

def multiplier(n):
    def multiply_by(x):
        return x * n
    return multiply_by

times_three = multiplier(3)
print(times_three(5))  # Output: 15

• multiplier(3) returns a closure multiply_by that remembers n = 3.
• When times_three(5) is called, it multiplies 5 by 3.

Checking a Closure in Python

• Closures store the variables they reference from their outer function in the __closure__ attribute.
• They help avoid global variables while maintaining state across function calls.

def outer():
    x = 10
    def inner():
        print(x)
    return inner

closure_func = outer()
print(closure_func.__closure__)  # Output: (<cell at 0x...: int object at 0x...>,)

#7 Single Underscore Variable: `_`

The single underscore _ is used in Python for different purposes:

• Temporary or unimportant variable: _ is used as a placeholder when a variable is needed, but its value doesn't matter.
• Last result in the Python REPL: _ stores the result of the last expression you evaluated.
• Ignoring values: In loops or unpacking, _ is used to skip values you don't need.

#### Temporary variable
for _ in range(3):
    print("Hello!")  # Output: Hello! (printed 3 times)

#### Last result in the REPL
>>> 5 + 3
8
>>> _ * 2
16

#### Ignoring Values
coordinates = (10, 20, 30)
x, _, z = coordinates

print(x)  # Output: 10
print(z)  # Output: 30

#8 Difference between static and class methods

• Static Methods (@staticmethod): These methods don’t need access to the class or instance. They’re used to keep things organized within the class and can be called with either the class name or an instance.
• Class Methods (@classmethod): These methods get the class as the first argument (cls) and can change the class state. They’re useful for creating instances in specific ways or altering the class itself.

#9 __getattr__() and __dir__() functions

In Python, you can customize how attributes are accessed in a class by defining the __getattr__() and __dir__() methods. They also can be used in module level.

• __getattr__(): This method is called when an attribute isn't found. It can be used to handle missing attributes or generate values dynamically. For example, if you look for an attribute that doesn’t exist, __getattr__() can respond or raise an error.
• __dir__(): This method is used when you call dir() on an object. It should return a list of attribute names for that object.

from typing import Any
from warnings import warn

def new_func(d: dict[str, Any], key: str) -> Any:
    ...

def __getattr__(name: str):
    if name == "old_func":
        warn(f"{name} is deprecated", DeprecationWarning)
        return new_func
    raise AttributeError(f"module {__name__} has no attribute {name}")

#10 Generators

Generators offer a neat way to create functions that return a sequence of values. Using the yield statement, a generator can pause and return a value, then resume where it left off, saving its state between calls.

You can get new values from generators just like iterators, using the next() function or loops.

Here’s an example with a generator that produces Fibonacci numbers:

def fibonacci():
    a, b = 0, 1
    while True:
        yield b
        a, b = b, a + b

# Create a generator object
fib = fibonacci()

# Retrieve values from the generator
print(next(fib))  # Output: 1
print(next(fib))  # Output: 1
print(next(fib))  # Output: 2
print(next(fib))  # Output: 3

In this example:

• The fibonacci() function creates a generator that yields Fibonacci numbers.
• Each call to next(fib) gives the next number in the sequence, with the generator keeping track of where it left off.

#11 Iterators

An iterator is an object that implements two methods: __iter__() and __next__(). Generators are a type of iterator, but you can also create custom iterators.

class MyIterator:
    def __init__(self, data):
        self.data = data
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.index >= len(self.data):
            raise StopIteration
        value = self.data[self.index]
        self.index += 1
        return value

# Create an instance of MyIterator
my_iter = MyIterator([1, 2, 3])

# Use the iterator in a for loop
for item in my_iter:
    print(item)

In this example:

• MyIterator is a custom iterator class that iterates over a list.
• The __iter__() method returns the iterator object itself.
• The __next__() method returns the next value and raises StopIteration when there are no more values.

This custom iterator can be used in a for loop just like built-in iterators.

#12 Decorators

A decorator is a function that takes another function and adds extra behavior to it without changing the function’s code. They are commonly used for tasks like logging, access control, and caching.

def my_decorator(func):
    def wrapper():
        print("Something before the function")
        func()
        print("Something after the function")
    return wrapper

@my_decorator
def say_hello():
    print("Hello!")

say_hello()
# Output:
# Something before the function
# Hello!
# Something after the function

#13 Context Managers

Context managers help manage resources like files or database connections, making sure they are properly opened and closed. You usually use them with the with statement.

You can create custom context managers using the contextlib module or by defining __enter__ and __exit__ methods.

from contextlib import contextmanager

@contextmanager
def my_context():
    print("Entering context")
    yield
    print("Exiting context")

with my_context():
    print("Inside context")


#### file processing with builtin context managers

with open('file.txt', 'r') as file:
    content = file.read()
# The file is automatically closed after the block

#14 Monkeypatching

Monkey patching allows you to modify or extend existing classes, modules, or objects at runtime without changing their original source code. It can dynamically alter their behavior to suit specific needs.

• Risks: Monkey patching can cause difficult-to-find bugs and might break code if not done carefully.
• Maintenance: It can make the code harder to understand and maintain, especially with multiple patches.

When to Use Monkey Patching

• Testing: To mock or replace functions or methods during tests.
• Bug Fixes: To quickly fix issues or add workarounds in third-party libraries.
• Adding Features: To add new methods or change behavior in external code.

# Original class
class Dog:
    def bark(self):
        return "Woof!"

# Monkey patching the bark method
def new_bark():
    return "Meow!"

dog = Dog()
print(dog.bark())  # Output: "Woof!"

# Applying the monkey patch
dog.bark = new_bark
print(dog.bark())  # Output: "Meow!"

#15 Enumerator

In Python, enumerate() is a built-in function that allows you to iterate over a sequence (like a list, tuple, or string) while keeping track of the index position of each element in the sequence. It returns an enumerate object, which produces pairs containing an index and the corresponding item from the sequence.

enumerate(iterable, start=0) # basic syntax of enumerate


### Example
fruits = ['apple', 'banana', 'cherry']

for index, fruit in enumerate(fruits):
    print(index, fruit)

### Output
> 0 apple
> 1 banana
> 2 cherry

#16 Pythonic way to Getters and Setters

The @property decorator lets you define methods that act like attributes, providing a clean way to manage attribute access and modification. The @attribute_name.setter decorator is used to set a method that will handle changes to the attribute. This approach keeps the code clean and allows for easy data validation.

class Circle:
    def __init__(self, radius: int):
        self._radius = radius

    @property
    def radius(self):
        return self._radius

    @radius.setter
    def radius(self, value: int):
        if value < 0:
            raise ValueError("Radius cannot be negative!")
        self._radius = value

# Example usage
circle = Circle(10)
print(f"Initial radius: {circle.radius}")  # Output: Initial radius: 10

circle.radius = 15
print(f"New radius: {circle.radius}")  # Output: New radius: 15

# Trying to set a negative radius will raise an error
# circle.radius = -5  # Uncommenting this line will raise a ValueError

• @property makes the radius method act like an attribute getter.
• @radius.setter allows validation and modification of the radius attribute.

#17 Shallow Copy and Deep Copy

In Python, shallow copy and deep copy are two methods to duplicate objects, but they handle nested objects differently.

Choosing the right method depends on whether you need just a surface-level copy or a complete, independent copy of the entire object structure.

Shallow Copy

• What It Does: Creates a new object but only copies references to nested objects, not the nested objects themselves.
• Effect: Changes to nested objects affect both the original and the copy since they share the same references.

import copy

original_list = [[1, 2, 3], [4, 5, 6]]
shallow_copy = copy.copy(original_list)

shallow_copy[0][0] = 'X'
print(original_list)  # Output: [['X', 2, 3], [4, 5, 6]]
print(shallow_copy)   # Output: [['X', 2, 3], [4, 5, 6]]

Use When: You need a copy of the outer object but don’t need to independently modify nested objects.

Deep Copy

• What It Does: Creates a new object and recursively copies all nested objects, so the entire structure is duplicated.
• Effect: Changes to the copied object do not affect the original object since all nested objects are independently copied.

import copy

original_list = [[1, 2, 3], [4, 5, 6]]
deep_copy = copy.deepcopy(original_list)

deep_copy[0][0] = 'X'
print(original_list)  # Output: [[1, 2, 3], [4, 5, 6]]
print(deep_copy)      # Output: [['X', 2, 3], [4, 5, 6]]

• Use When: You have complex objects with nested structures and need a fully independent copy.

#18 Metaclasses

A metaclass is like a “class of classes.” It defines how other classes are created and behave. You can use metaclasses to control class creation, modification, and behavior.

Example: Singleton Pattern with a Metaclass

The Singleton pattern ensures that a class has only one instance and provides a global point of access to that instance. You can implement this pattern using a metaclass.

class SingletonMeta(type):
    _instances = {}
    
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class SingletonClass(metaclass=SingletonMeta):
    def __init__(self, value):
        self.value = value

# Example usage
singleton1 = SingletonClass('First Instance')
singleton2 = SingletonClass('Second Instance')

print(singleton1.value)  # Output: First Instance
print(singleton2.value)  # Output: First Instance
print(singleton1 is singleton2)  # Output: True

• SingletonMeta: The metaclass with a __call__ method that controls object creation. It ensures only one instance of the class is created.
• SingletonClass: A class that uses SingletonMeta as its metaclass. It will only ever have one instance.

In this example:

• singleton1 and singleton2 are the same instance, even though we try to create a second one.
• The metaclass manages the instance, ensuring only one exists.

Additional Resources for Deep Diving

Expert Python Programming - Fourth Edition

Effective Python: 90 Specific Ways to Write Better Python, 2nd Edition

Mastering Python Design Patterns - Third Edition

Finally

Hopefully, it is helpful to understand some of lesser-known concepts of Python.

Image generated by AI.

You can find my links below to follow me on other platforms.

Kind regards

@mebaysan - IT Professional

BaysanSoft

mebaysan - Overview