Python: Super() & MRO
Co-Author: Y. S. Huang, a master’s student studying AIVC, likes open-source.
If you are interested, go to check my Github!
We’ve introduced the inheritance syntax in the former article. We mentioned that Super() represents the base class. Super directs to the base class and can execute a specific method of it. However, when you go deep into Super, you’ll find that it’s associated with MRO(Method Resolution Order).
Contents
New-Style and Old-Style Class
Before we start to talk about MRO, it’s necessary to review the history of the class in python.
#old-style class (before python 2.1
class A:
def __init__(self):
pass
#new-style class (after python 2.2
class A(object):
def __init__(self):
pass
You can see that the latter one inherits object
. In python 3.X, the old-style class had been abandoned. whichever way you define a class, python 3 sees it as new-style class. Here we’re introducing the concept of old-style and new-style for your better understanding in the following article.
MRO
Method Resolution Order(MRO) is used in conditions of multi-inheritance. It defines the order that the interpreter analyzes functions. Let’s see an example code to explain it.
class A():
def who_am_i(self):
print("I am A")
class B(A):
pass
class C(A):
def who_am_i(self):
print("I am C")
class D(B,C):
pass
d = D()
#d.who_am_i()
It’s a simple code with class, but can you tell the result if d.who_am_i()
is added to it? Before we show the answer, I’d like to explain the difference between old-style and new-style MRO.
The old-style MRO will follow the order of inheritance and go through each branch. As a result, the order of the old-style MRO in this code will be DBACA. However, one class can only show up once. It turns out that the latter ‘A’ will be removed. The order thus becomes DBAC.
Some defects are found in old-style MRO. Consequently, in Python 3, it comes out with a new MRO algorithm.
The new-style MRO is similar to the old one. They have the same search method. The main difference is that the new one has an additional check in the candidate-path. The interpreter checks whether the branch point is good. If there is no duplicate node, then it is a good inherit-node; otherwise, it will be removed from the inherit-order.
Take the picture above as an example. The path is DBACA, and the interpreter finds that A is not a good branch point because C inherited the latter A. Therefore, the former A has been removed, and the path order turned out to be DBCA. This is the MRO in python 2.2.
Now let’s see an example code to understand it better.
class Animal(object):
def __init__(self, Animal):
print(Animal, 'is an animal.')
class Mammal(Animal):
def __init__(self, mammalName):
print(mammalName, 'is a warm-blooded animal.')
super(Mammal, self).__init__(mammalName)
class NonWingedMammal(Mammal):
def __init__(self, mammalCannotFly):
print(mammalCannotFly, "can't fly.")
super(NonWingedMammal, self).__init__(mammalCannotFly)
class NonMarineMammal(Mammal):
def __init__(self, mammalCannotSwim):
print(mammalCannotSwim, "can't swim.")
super(NonMarineMammal, self).__init__(mammalCannotSwim)
class Dog(NonMarineMammal, NonWingedMammal):
def __init__(self, mammalName):
print('Dog has 4 legs.')
super(Dog, self).__init__(mammalName)
D = Dog('Dog')
#print(Dog.__mro__)
#---output--- CPython 2.2.3#
Dog has 4 legs.
('Dog', 'cant swim.')
('Dog', 'cant fly.')
('Dog', 'is a warm-blooded animal.')
('Dog', 'is an animal.')
Here we create an object with the class Dog
named ‘d’. In its __init__
method, it uses super(Dog, self)__init__(mammalName)
to call the parent class method. We know that the leftmost parameter has the top priority in multiple inheritances. Thus it goes into classNonMarineMammal
. ClassNonMarineMammal
also calls super(NonMarineMammal, self)__init__(mammalCannotSwim)
, Here is the most important. ClassNonMarineMammal
and NonWingedMammal
each inherits the class Mammal
. It indicates that the former branch point classMammal
is not good and will be removed. The path thus goes to the next class NonWingedMammal
. The NonWingedMammal
follows the super() to the Mammal.
ClassMammal
then does the same thing and goes to class Animal
. You can also use Dog.__mro__
to check the answer.
print(Dog.__mro__)
#---output--- CPython 2.2.3#
(<class '__main__.Dog'>,
<class '__main__.NonMarineMammal'>,
<class '__main__.NonWingedMammal'>,
<class '__main__.Mammal'>,
<class '__main__.Animal'>,
<type 'object'>)
The MRO ensures that in the use of multi-inheritance, you won’t encounter the condition of repeated execution.
However, the new-style MRO still has defects. It is possible to obey the rule of Monotonicity. The rule of Monotonicity is that in multiple inheritances, the subclass cannot change the order of the base class’s inherit-order. Otherwise, it causes the error. Let’s see an example.
class X(object):
pass
class Y(object):
pass
class A(X,Y):
pass
class B(Y,X):
pass
class C(A, B):
pass
For class A
, the inherit-order is “A->X->Y->object”. For class B
, the inherit-order is “B->Y->X->object”. For class C
, the inherit-order is “C->A->B->X->Y->object”. You can see that the candidate-path of class B
and class C
are different. When the class B
is inherited, the inherit-path of itself is changed, which obeys the Monotonicity rule.
In python 2.3, with the C3 Algorithm, when executing the code above, it results in the error:
#---output--- CPython 2.2.3#
Traceback (most recent call last):
File "/home/jason/Desktop/python2_test/test.py", line 9, in <module>
class C(A, B):
TypeError: Error when calling the metaclass bases
Cannot create a consistent method resolution
order (MRO) for bases Y, X
You can only resolve this problem by manually changing the order.
C3 Superclass Linearization
The MRO before python 2.3 used Depth-First-Search as its algorithm. This algorithm has a defect in obeying the local Precedence and Monotonicity rule. C3 algorithm was introduced into python to resolve these problems. Monotonicity has been introduced in the above. The rule of Local Precedence is that class C inherits A and B like:
class C(A, B)
When checking the attributes of C, it accords with the declaration order. In this case, it first goes to A
then B
.
The MRO problem existing before python 2.3 can be solved with the C3 algorithm. Before we start, here are some math-marks you need to know first.
__mro__
is a list of the correct inherit-order.- H means the candidate becomes the next inherit node for the iteration, and the candidate is always the first element in the list.
- L(K) means the linearization of K, which:
L(K) = [K] + merge(L(pre-class), [pre-class])
L(K) = [K], if pre-class is not exist merge()
is the way of C3 calculation.
For linearization, let’s take an Object as the first example and take that O
is inherited by A
as the second example.
class O
class A(O)
L(O) = [O] + merge(L(object), [object])
= [O] + merge([object], [object])
= [O, object]
L(A) = [A] + merge(L(O), [O])
= [A] + merge([O, object], [O])
= [A, O] + merge([object])
= [A, O, object]
Merge is a function. We use merge to keep the consistency. The exact process of merge()
is as follows:
- Mark the first element in the list as H.
- It starts from the first list in Merge. If H does not exist in other lists or only in the other lists by the first element, then it is a good candidate. Otherwise, it is not.
- If H is a good candidate, we put H into the
__mro__
, which is the list in the beginning, and delete the element H from all the lists inmerge()
, and let the first element in the first list of themerge()
become the H, then jump to Step 2. - If H is not a good one, let the first element in the next list of the
merge()
become H, then jump to Step 2. - Repeat those steps until the merge() is empty.
For example:
class O
class A(O)
class B(O)
class C(O)
class K1(A, B, C)
class K2(D, B, E)
class K3(D, A)
class Z(K1, K2, K3)
L(K1) = [K1] + merge(L(A), L(B), L(C), [A, B, C]) // (1)
= [K1] + merge([A, O], [B, O], [C, O], [A, B, C]) // (2)
= [K1, A] + merge([O], [B, O], [C, O], [B, C]) // (3)
= [K1, A, B] + merge([O], [O], [C, O], [C]) // (4)
= [K1, A, B, C] + merge([O], [O], [O]) // (5)
= [K1, A, B, C, O]
L(K2) = [K2, D, B, E, O]
L(K3) = [K3, D, A, O]
L(Z) = [Z] + merge(L(K1), L(K2), L(K3), [K1, K2, K3])
= [Z] + merge([K1, A, B, C, O], [K2, D, B, E, O], [K3, D, A, O], [K1, K2, K3])
= [Z, K1] + merge([A, B, C, O], [K2, D, B, E, O], [K3, D, A, O], [K2, K3])
= [Z, K1, K2] + merge([A, B, C, O], [D, B, E, O], [K3, D, A, O], [K3])
= [Z, K1, K2, K3] + merge([A, B, C, O], [D, B, E, O], [D, A, O])
= [Z, K1, K2, K3, D] + merge([A, B, C, O], [B, E, O], [A, O])
= [Z, K1, K2, K3, D, A] + merge([B, C, O], [B, E, O], [O])
= [Z, K1, K2, K3, D, A, B] + merge([C, O], [E, O], [O])
= [Z, K1, K2, K3, D, A, B, C] + merge([O], [E, O], [O])
= [Z, K1, K2, K3, D, A, B, C, E] + merge([O], [O], [O])
= [Z, K1, K2, K3, D, A, B, C, E, O]
- In (1), we transform L(A), L(B), L(C) into the form of their linearization.
- In (2), A is a good candidate because it only exists in the first list and the first element of the last list. So we put it into the
__mro__
and deleted it from the lists in themerge()
. - In(3), O is not a good candidate because it is the second element in the 2nd & 3rd list. So we move on to the second list [B, O]. B is a good candidate just like (2).
- In(4), O is not a good candidate because it exists in the third list. C is a good one.
- In(5), O is the last element, and we put it into the
__mro__
.
This way, you understand the process of C3 algorithm. The following class K2
, K3
and Z
adhere to the same rule. To better understand the complexity of C3 algorithm, __mro__
of the class Z
is expanded.
C3 algorithm is better than the Old-Style MRO and New-Style MRO, for it realizes the characteristic of Local Precedence and Monotonicity.
Tricky Super
There are some tricky things you may have wondered about when using super()
. Take a look at the code below.
class Base(object):
def __init__(self):
print("Base created")
class Child_A(Base):
def __init__(self):
super().__init__()
class Child_B(Base):
def __init__(self):
Base.__init__(self)
Child_A()
Child_B()
# ---output---
Base created
Base created
Child_A
and Child_B
can both initialize objects. However, you may wonder what is the difference between super().__init__()
and Base.__init__(self)
?
In Child_A
you get an indirect relation to the __init__()
with super()
, which uses the class defined to determine the next class's __init__()
to locate it in the __mro__
.
The other uses the BASE
direct, calling the parent class’s method.
Another error you might encounter in MRO is the use of super.
class AA:
def __init__(self) -> None:
self.aa = 'aa'
#super().__init__() without this line, AA won't pass self and execuate __init__() to the next inherit-order
class BB:
def __init__(self) -> None:
super().__init__()
self.bb = 'bb'
class CC(AA, BB):
def __init__(self) -> None:
super().__init__()
dd = CC()
print(CC.__mro__)
print(dd.aa)
print(dd.bb)
---output---
(<class '__main__.DD'>, <class '__main__.AA'>, <class '__main__.BB'>, <class 'object'>)
aa
#AttributeError: 'DD' object has no attribute 'bb'
Look at the class AA
. If super().__init__()
is not written, it won’t use self
go through the next inherit-order, which is class BB
. Then it turned out that the CC
has no attribute bb
. To fix this problem, make sure you add super()
in each class that is used in inheritance so that it enters the order in __mro__
.
In conclusion, the one used super()
has a specific order of parent class by MRO algorithm.
Thesuper()
lets you follow the __mro__
to inheritance classes correctly!