“I do not like to repeat successes, I like to go on to other things”.

Walt Disney

At the beginning of my way as a Python developer, I found myself searching for new bits of knowledge of what Python is capable of as a programming language. Digging too deep into details made me understand how things work “under the hood”. The more I learned, the more I had to google things a knew. It takes a while, so I started to make notes on the most interesting things.

After a while, since I started writing notes and scratches, I had hundreds of them. In this article, I share the most commonly used.

The very first thing to know before you can go any further in learning Python is the way to reach the Zen of Python. It is a collection of 19 software principles that influence the design of Python Programming Language and is included as an Easter egg in the Python interpreter, which can be displayed by entering import this.

>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
...

This is an elegant way of initializing and generating lists. The official documentation describes the full set of possibilities.

Example:

>>> squares = [x**2 for x in range(10)]
>>> squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]

Python 3.5+ provides an easy way of how to merge dictionaries. The official documentation about dictionaries.

1. Using {**kwargs}

>>> x = {‘a’: 1, ‘b’: 2}
>>> y = {‘b’: 3, ‘c’: 4}
>>> z = {**x, **y}>>> z{‘a’: 1, ‘b’: 3, ‘c’: 4}

2. Calling update. Example:

>>> dict1 = {'a': 1, 'b': 2}
>>> dict2 = {'d': 4, 'c': 4}
>>> dict2.update(dict1)
{'d': 3, 'c': 4, 'a': 1, 'b': 2}

The first way is to write your own way to get a value from a dictionary

>>> xs = {'a': 4, 'b': 3, 'c': 2, 'd': 1}
>>> sorted(xs.items(), key=lambda x: x[1])
[('d', 1), ('c', 2), ('b', 3), ('a', 4)]

Another way is to use operator.itemgetter

>>> xs = {'a': 4, 'b': 3, 'c': 2, 'd': 1}
>>> import operator
>>> sorted(xs.items(), key=operator.itemgetter(1))
[('d', 1), ('c', 2), ('b', 3), ('a', 4)]

The first way to do this is a simple broadly known if/else statement

def dispatch_if(operator, x, y):
if operator == 'add':
return x + y
elif operator == 'sub':
return x - y
elif operator == 'mul':
return x * y
elif operator == 'div':
return x / y
else:
return None

Another way to do the same is to use a dictionary

def dispatch_dict(operator, x, y):
return {
'add': lambda: x + y,
'sub': lambda: x - y,
'mul': lambda: x * y,
'div': lambda: x / y,
}.get(operator, lambda: None)()

Default values for Namedtuples

Instantiating Namedtuples classes

This section will begin with samples. Here I am going to dive straight in. You may find some methods you already met.

dct.__getitem__(key)
dct.__setitem__(key, value)
dct.__delitem__(key)
ins.__getattr__(attr)
ins.__setattr__(attr, value)
seq.__next__()

As you may have noticed “__” (two underscores) is also known as “dunder”.

These methods are called magical because you don’t need to call them explicitly. They are called via built-in operators or functions. The samples of operators follow methods each is calling.

dct.__getitem__(key)
>>> x[key]
dct.__setitem__(key, value)
>>> x[key]= value
dct.__delitem__(key)
>>> del x[key]
seq.__next__()
>>> next(seq)
ins.__getattr__(attr: str)
>>> ins.attr # here the instance attribute is retrieved. The 'attr' after a dot (".")is not a variable
ins.__setattr__(attr, value)
>>> ins.attr = value

By default, those methods are predefined in most of the cases. However, Python lets you override built-ins so you can write the code that fits your needs better. For a good example of how those dunder methods are used, you can visit agithub.

Now you know what the dunder methods are. One of the use cases for them is to make a class iterable. This kind of class allows you to iterate over its values. Imagine now how you would iterate over a “list items” just like:

for item in list

Let’s write our own iterable class with our own logic of iteration over it.

The class name would be IterableExercise. It has a list of exercises. We define __iter__ method. This iterates over the exercises and yields the tuple of lists of muscles you train with a body top.

class IterableExercise:
exercises = [
{'top': True, 'core': False, 'arms': ['biceps', 'triceps'], 'back': [], 'legs': []},
{'top': True, 'core': True, 'arms': [], 'back': ['trapeze'], 'legs': []},
{'top': False, 'core': True, 'arms': [], 'back': ['loin'], 'legs': ['quadriceps']},
]

def __iter__(self):
for exercise in self.exercises:
if exercise['top']:
yield exercise['arms'], exercise['back'], exercise['legs']
# Here is how you use it>>>for arms, back, legs in IterableExercise(): arms_text = ' '.join(str(e) for e in arms) back_text = ' '.join(str(e) for e in back) legs_text = ' '.join(str(e) for e in legs) print('Top body is working when you train %s %s %s' % (arms_text, back_text, legs_text))Top body is working when you train biceps tricepsTop body is working when you train trapeze

Beginning from Python 3.5 you can use a static typing. It is useful when you need to support a big project and it comes hard to keep the type of the arguments passed to the methods in mind. Just remember that there is some kind of type helper out there. You can use it like this:

def my_add(a: int, b: int) -> int:
return a + b

Dive into documentation for a better understanding of how it works.

Registry

This section describes the use of the Registry Pattern (RP) in a sample of Extract Transform Load (ETL) process. Most of the developers that worked with data processing and data manipulating may have faced the issue of custom modifications that are based on natural rules. RP is aimed to help make such modifications easier. The need for adding new modules will remain still, but yet, things do not have to be hard on every part of a project lifetime.

The RP provides global accessibility of registered services. Simple inheritance in Python allows building a Registry class. It can be a Metaclass or even a single base class beginning from Python 3.6.

Here, the examples of how Registry class can be built are provided. The way the RP is implemented in case of run-time registration allows new classes to be added to the project with no need to explicitly register them in multiple places. That makes a codebase easier to extend for future modifications.

For the sake of simplicity, the Django skeleton is used as a template for project structure. The following command generates the project structure you will further need to modify according to your needs.

$ django-admin startproject — template=https://github.com/Mischback/django-project-skeleton/archive/development.zip [projectname]

The project structure with our ETL module looks like this:

[projectname]/ <- project root
├── [projectname]/ <- Django root
│ ├── __init__.py
│ ├── settings/…
│ ├── urls.py
├── apps/
│ └── __init__.py
│ └── app_one/…
├── etl/
│ └── __init__.py
│ └── extractors.py
│ └── transformers.py
│ └── loaders.py
│ └── services/
│ │ └── __init__.py
│ │ └── service_one.py
│ │ └── service_two.py
│ │ └── service_three.py
├── manage.py
├── README.md
├── …

The etl module has a standard set of python files each responsible for data processing.

  1. Registry class
  2. Metaclass as Registry
  3. Registry of distributed classes in different files

The following code registers all its subclasses in the registry class property. It ignores base_classes while saving and warns if any subclass does not have an identifier_property.

import warnings


class RegistryHolder(type):
registry = {}
base_classes = ['BaseServiceOne', 'BaseServiceTwo', 'BaseServiceThree']

@classmethod
def register_class(mcs, target_class):
try:
identifier_property = getattr(target_class, 'identifier_property')
mcs._save(identifier_property, target_class)
except AttributeError:
mcs._warn(target_class.__name__)

def __new__(mcs, name, bases, class_dict):
cls = type.__new__(mcs, name, bases, class_dict)
mcs.register_class(cls)
return cls

@classmethod
def _save(mcs, identifier_property, target_class):
registered_class = mcs.registry.get(identifier_property)

if registered_class is None:
mcs.registry[identifier_property] = registered_class
else:
warning_message = 'Error occurred while registering {class_name}. identifier_property {identifier_property} is already registered for class {registered_class_name}' # noqa: E501
warnings.warn(warning_message.format(
identifier_property=identifier_property,
registered_class_name=registered_class.__name__,
class_name=target_class.__name__),
RuntimeWarning)

@classmethod
def _warn(mcs, class_name):
if class_name not in mcs.base_classes:
warnings.warn('{class_name} does not have attribute identifier_property'.format(class_name=class_name),
RuntimeWarning)

For as Python is a scripting language, only classes that are imported during the code execution will appear in the registry. This problem can be solved by auto-importing all the submodules in the parent package. Do it by adding the following code in init file in parent package:

def import_submodules(package, recursive=True):
""" Import all submodules of a module, recursively, including subpackages

:param recursive: a flag to import submodules recursively
:param package: package (name or actual module)
:type package: str | module
:rtype: dict[str, types.ModuleType]
"""
if isinstance(package, str):
package = importlib.import_module(package)
results = {}
for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
full_name = package.__name__ + '.' + name
results[full_name] = importlib.import_module(full_name)
if recursive and is_pkg:
results.update(import_submodules(full_name))
return results


import_submodules(__name__)

There is still a lot to learn out there, but now you have a starting point of a direction to dig in. Pick up a topic you may find useful and interesting, dive into the documentation and go into details. Try your best to understand how things actually work. I bet the result will bring you fulfillment and expand the knowledge that you already have! Even if you start with something small, you are heading in the right direction.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store