Cooperative multiple-inheritance paradigm in python

Ron Urbach
Jun 25, 2020 · 6 min read

The goal of this article is to raise the awareness for the cooperative multiple-inheritance paradigm in python. Not knowing about it could lead to bugs that are quite hard to investigate, and opposed to the usual pythonic zen, it hides some implicit behavior.

We’ll begin with an example for the bug that may raise, with it you’ll understand how this paradigm can meet your development. Afterwards, we’ll dive in and understand how, and when, should you embrace it into your code.

I assume the reader’s knowledge of python programming language. All snippets are written in with python3.5 syntax.

Let’s start with a simplified bug:

>> python3 cooperative.py
Traceback (most recent call last):
File "cooperative.py", line 20, in <module>
Tractor("yellow")
File "cooperative.py", line 17, in __init__
ConstructionMachine.__init__(self)
File "cooperative.py", line 8, in __init__
super().__init__()
TypeError: __init__() missing 1 required positional argument: 'color'

What?!

What really happened?

This is used when you call a method of some instance — at first the method is looked in the instance’s class, but if not found then python looks up the inheritance graph (by that determined order). This order is determined during runtime, and you can access it. The C3 linearization algorithm promises a monotonic MRO.

The MRO for Tractor is:

>> Tractor.mro() 
Out[2]: [__main__.Tractor, __main__.ConstructionMachine, __main__.Vehicle, object]

And the MRO for ConstructionMachine is:

>> ConstructionMachine.mro()
Out[3]: [__main__.ConstructionMachine, object]

The key issue here is that super() finds the parent according to the current MRO. So if we were looking for method rumble() in class Tractor (which doesn’t have it directly but it inherits it from ConstructionMachine), python will go over the MRO and find it. But then notice line 11, inside rumble() : a call for super() is used, and although the MRO for ConstructionMachine has no farther parent, the MRO that will be used is the current one (from Tractor !), which has Vehicle class afterwards. So ConstructionMachine’s rumble() will in fact call Vehicle.__str__ when called from a Tractor instance.

In our bug, __init__ is the method which is called with Tractor’s MRO and accordingly when we reach the ConstructionMachine.__init__ the super() call then triggers Vehicle.__init__ , that expects a “color” argument.

But why call super from __init__ if you have no parent?

Diamond Problem

For example, if Vehicle and ConstructionMachine will inherit Machine (as illustrated in the UML graph), then when Tractor is constructed it will explicitly call for Vehicle.__init__ and also for ConstructionMachine.__init__ , where both of these functions will explicitly call for Machine.__init__ since they should explicitly construct their own parent (which is Machine for both) hence Machine is constructed twice.

Clearly this is only one example out of many under this domain, and that’s why super() had been introduced. Since super() goes over the MRO and the MRO doesn’t contain duplicates, it solves the problem. However, for this solution to happen all classes must support the cooperative multi-inheritance paradigm. In a nutshell, all classes must call super().__init__() (once) from their __init__s instead of explicitly calling the __init__s of their parents, and additionally forward the constructor’s params by **kwargs. Not only for init, but for every function that is used via multiple-inheritance. This is not the whole idea but we’ll deal with that later.

It can be thought of as taking a parcel of all the arguments for all the classes-constructors along the inheritance graph, where each class takes what it needs from that parcel and forwards it to the following class according to the MRO. Ugly, but necessary — and thus not very adequate to python’s zen.

To summarize, in our bug example ConstructionMachine is aiming to be a cooperative class and that’s why it has the super() call in it’s __init__ although it doesn’t have a parent, which finally causes the bug.

Could this happen in real life?

When having both cooperative classes and non-cooperative classes work together collisions as this are occurring and such bugs are hard to identify if you’re not aware to these issues, as the program crashes and the trace-back is surprisingly pointing to an almost non-related code section.

Not only PyQt uses this. Every python package that follows this paradigm must declare it in its documentation.

How to follow the paradigm?

All classes that has some kind of inheritance relationship with one another should follow the cooperative paradigm. Let’s focus on the constructors of cooperative classes:

  1. Every __init__ should accept **kwargs in addition to its arguments. E.g. def __init__(self, arg_a: int, arg_b: int, **kwargs).
  2. Every call to a cooperative class constructor must be done with non-positional keyword-ed arguments. E.g. a=A(arg_a=14, arg_b=0) .
  3. Every __init__ should call to the super’s init, and exactly once. When doing so, it must pass the **kwargs onward, but also add the variables it used back into the **kwargs parcel. E.g. self._a=arg_a; kwargs[arg_a] = arg_a; super().__init__(**kwargs).

Now constructors are covered, and if there is a common argument name in multiple constructors it isn’t wrapped away. The keyworded call assures no positional conflicts and the single call for super guarantees that all classes along the MRO will be initialized.

The same rules can be used for all other overridden methods, even custom ones, but for that one might need to inherit from a custom “Root” class that wraps it around.

I strongly urge the reader to learn from the following article: super considered super — by Raymond Hettinger, which covers more edge cases such as how to wrap 3rd party libraries that are not cooperative to align with your cooperative classes, and more. Note that this article does not deal with common argument names, but the solution for this problem was described above.

Should this paradigm be embraced?

With that, I find it utterly important to be aware of this paradigm. One can still use packages that follow the cooperative multiple inheritance paradigm with others that don’t, while avoiding the diamond-problem by designing the code that way, and just solve this potential problem as it’s being encountered. Or, at least suspect it when something weird happens, and then play with the inheritance declaration order (the tuple of a class’s __bases__) to see if a different MRO will solve such issues.

The Startup

Get smarter at building your thing. Join The Startup’s +792K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +792K followers.

Ron Urbach

Written by

Software Developer

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +792K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store