Python: Super() & MRO

Co-Author: Y. S. Huang, a master’s student studying AIVC, likes open-source.
If you are interested, go to check my Github!

We’ve introduced the inheritance syntax in the former article. We mentioned that Super() represents the base class. Super directs to the base class and can execute a specific method of it. However, when you go deep into Super, you’ll find that it’s associated with MRO(Method Resolution Order).

Python: Advanced Syntax of Class

Contents

• New-Style and Old-Style Class
• MRO
• C3 Superclass Linearization
• Tricky Super

New-Style and Old-Style Class

Before we start to talk about MRO, it’s necessary to review the history of the class in python.

#old-style class (before python 2.1
class A:
    def __init__(self):
        pass
#new-style class (after python 2.2
class A(object):
    def __init__(self):
        pass

You can see that the latter one inherits object . In python 3.X, the old-style class had been abandoned. whichever way you define a class, python 3 sees it as new-style class. Here we’re introducing the concept of old-style and new-style for your better understanding in the following article.

MRO

Method Resolution Order(MRO) is used in conditions of multi-inheritance. It defines the order that the interpreter analyzes functions. Let’s see an example code to explain it.

class A():
    def who_am_i(self):
        print("I am A")
        
class B(A):
    pass
        
class C(A):
    def who_am_i(self):
        print("I am C")

class D(B,C):
    pass
    
d = D()
#d.who_am_i()

It’s a simple code with class, but can you tell the result if d.who_am_i() is added to it? Before we show the answer, I’d like to explain the difference between old-style and new-style MRO.

The old-style MRO will follow the order of inheritance and go through each branch. As a result, the order of the old-style MRO in this code will be DBACA. However, one class can only show up once. It turns out that the latter ‘A’ will be removed. The order thus becomes DBAC.

Some defects are found in old-style MRO. Consequently, in Python 3, it comes out with a new MRO algorithm.

The new-style MRO is similar to the old one. They have the same search method. The main difference is that the new one has an additional check in the candidate-path. The interpreter checks whether the branch point is good. If there is no duplicate node, then it is a good inherit-node; otherwise, it will be removed from the inherit-order.

Take the picture above as an example. The path is DBACA, and the interpreter finds that A is not a good branch point because C inherited the latter A. Therefore, the former A has been removed, and the path order turned out to be DBCA. This is the MRO in python 2.2.

Now let’s see an example code to understand it better.

class Animal(object):
    def __init__(self, Animal):
        print(Animal, 'is an animal.')

class Mammal(Animal):
    def __init__(self, mammalName):
        print(mammalName, 'is a warm-blooded animal.')
        super(Mammal, self).__init__(mammalName)

class NonWingedMammal(Mammal):
    def __init__(self, mammalCannotFly):
        print(mammalCannotFly, "can't fly.")
        super(NonWingedMammal, self).__init__(mammalCannotFly)

class NonMarineMammal(Mammal):
    def __init__(self, mammalCannotSwim):
        print(mammalCannotSwim, "can't swim.")
        super(NonMarineMammal, self).__init__(mammalCannotSwim)

class Dog(NonMarineMammal, NonWingedMammal):
    def __init__(self, mammalName):
        print('Dog has 4 legs.') 
        super(Dog, self).__init__(mammalName)
        
D = Dog('Dog')
#print(Dog.__mro__)
#---output--- CPython 2.2.3#
Dog has 4 legs.
('Dog', 'cant swim.')
('Dog', 'cant fly.')
('Dog', 'is a warm-blooded animal.')
('Dog', 'is an animal.')

Here we create an object with the class Dog named ‘d’. In its __init__ method, it uses super(Dog, self)__init__(mammalName) to call the parent class method. We know that the leftmost parameter has the top priority in multiple inheritances. Thus it goes into classNonMarineMammal. ClassNonMarineMammal also calls super(NonMarineMammal, self)__init__(mammalCannotSwim), Here is the most important. ClassNonMarineMammaland NonWingedMammal each inherits the class Mammal. It indicates that the former branch point classMammal is not good and will be removed. The path thus goes to the next class NonWingedMammal . The NonWingedMammal follows the super() to the Mammal. ClassMammal then does the same thing and goes to class Animal. You can also use Dog.__mro__ to check the answer.

print(Dog.__mro__)
#---output--- CPython 2.2.3#
(<class '__main__.Dog'>, 
 <class '__main__.NonMarineMammal'>,
 <class '__main__.NonWingedMammal'>,
 <class '__main__.Mammal'>,
 <class '__main__.Animal'>,
 <type 'object'>)

The MRO ensures that in the use of multi-inheritance, you won’t encounter the condition of repeated execution.

However, the new-style MRO still has defects. It is possible to obey the rule of Monotonicity. The rule of Monotonicity is that in multiple inheritances, the subclass cannot change the order of the base class’s inherit-order. Otherwise, it causes the error. Let’s see an example.

class X(object):
  pass
class Y(object):
  pass
class A(X,Y):
  pass
class B(Y,X):
  pass
class C(A, B):
  pass

For class A, the inherit-order is “A->X->Y->object”. For class B, the inherit-order is “B->Y->X->object”. For class C, the inherit-order is “C->A->B->X->Y->object”. You can see that the candidate-path of class B and class C are different. When the class B is inherited, the inherit-path of itself is changed, which obeys the Monotonicity rule.

In python 2.3, with the C3 Algorithm, when executing the code above, it results in the error:

#---output--- CPython 2.2.3#
Traceback (most recent call last):
  File "/home/jason/Desktop/python2_test/test.py", line 9, in <module>
    class C(A, B):
TypeError: Error when calling the metaclass bases
    Cannot create a consistent method resolution
order (MRO) for bases Y, X

You can only resolve this problem by manually changing the order.

C3 Superclass Linearization

The MRO before python 2.3 used Depth-First-Search as its algorithm. This algorithm has a defect in obeying the local Precedence and Monotonicity rule. C3 algorithm was introduced into python to resolve these problems. Monotonicity has been introduced in the above. The rule of Local Precedence is that class C inherits A and B like:

class C(A, B)

When checking the attributes of C, it accords with the declaration order. In this case, it first goes to A then B.

The MRO problem existing before python 2.3 can be solved with the C3 algorithm. Before we start, here are some math-marks you need to know first.

1. __mro__ is a list of the correct inherit-order.
2. H means the candidate becomes the next inherit node for the iteration, and the candidate is always the first element in the list.
3. L(K) means the linearization of K, which:
   L(K) = [K] + merge(L(pre-class), [pre-class])
L(K) = [K], if pre-class is not exist
4. merge() is the way of C3 calculation.

For linearization, let’s take an Object as the first example and take that O is inherited by A as the second example.

class O
class A(O)

L(O)  = [O] + merge(L(object), [object])
      = [O] + merge([object], [object])
      = [O, object]

L(A)  = [A] + merge(L(O), [O])
      = [A] + merge([O, object], [O])
      = [A, O] + merge([object])
      = [A, O, object] 

Merge is a function. We use merge to keep the consistency. The exact process of merge() is as follows:

1. Mark the first element in the list as H.
2. It starts from the first list in Merge. If H does not exist in other lists or only in the other lists by the first element, then it is a good candidate. Otherwise, it is not.
3. If H is a good candidate, we put H into the __mro__, which is the list in the beginning, and delete the element H from all the lists in merge(), and let the first element in the first list of the merge() become the H, then jump to Step 2.
4. If H is not a good one, let the first element in the next list of the merge() become H, then jump to Step 2.
5. Repeat those steps until the merge() is empty.

For example:

class O
class A(O)
class B(O)
class C(O)
class K1(A, B, C)
class K2(D, B, E)
class K3(D, A)
class Z(K1, K2, K3) 


 L(K1)  = [K1] + merge(L(A), L(B), L(C), [A, B, C])          // (1)
        = [K1] + merge([A, O], [B, O], [C, O], [A, B, C])    // (2)
        = [K1, A] + merge([O], [B, O], [C, O], [B, C])       // (3)
        = [K1, A, B] + merge([O], [O], [C, O], [C])          // (4)
        = [K1, A, B, C] + merge([O], [O], [O])               // (5)
        = [K1, A, B, C, O]

L(K2)   = [K2, D, B, E, O]

L(K3)   = [K3, D, A, O] 

 L(Z)   = [Z] + merge(L(K1), L(K2), L(K3), [K1, K2, K3])
        = [Z] + merge([K1, A, B, C, O], [K2, D, B, E, O], [K3, D, A, O], [K1, K2, K3])   
        = [Z, K1] + merge([A, B, C, O], [K2, D, B, E, O], [K3, D, A, O], [K2, K3])      
        = [Z, K1, K2] + merge([A, B, C, O], [D, B, E, O], [K3, D, A, O], [K3])          
        = [Z, K1, K2, K3] + merge([A, B, C, O], [D, B, E, O], [D, A, O])                 
        = [Z, K1, K2, K3, D] + merge([A, B, C, O], [B, E, O], [A, O])                   
        = [Z, K1, K2, K3, D, A] + merge([B, C, O], [B, E, O], [O])                      
        = [Z, K1, K2, K3, D, A, B] + merge([C, O], [E, O], [O])                         
        = [Z, K1, K2, K3, D, A, B, C] + merge([O], [E, O], [O])                         
        = [Z, K1, K2, K3, D, A, B, C, E] + merge([O], [O], [O])                         
        = [Z, K1, K2, K3, D, A, B, C, E, O]                                              

1. In (1), we transform L(A), L(B), L(C) into the form of their linearization.
2. In (2), A is a good candidate because it only exists in the first list and the first element of the last list. So we put it into the __mro__ and deleted it from the lists in the merge().
3. In(3), O is not a good candidate because it is the second element in the 2nd & 3rd list. So we move on to the second list [B, O]. B is a good candidate just like (2).
4. In(4), O is not a good candidate because it exists in the third list. C is a good one.
5. In(5), O is the last element, and we put it into the __mro__.

This way, you understand the process of C3 algorithm. The following class K2, K3 and Z adhere to the same rule. To better understand the complexity of C3 algorithm, __mro__ of the class Z is expanded.

C3 algorithm is better than the Old-Style MRO and New-Style MRO, for it realizes the characteristic of Local Precedence and Monotonicity.

C3 linearization - Wikipedia

Python MRO方法解析顺序详解

Tricky Super

There are some tricky things you may have wondered about when using super(). Take a look at the code below.

class Base(object):
    def __init__(self):
        print("Base created")
        
class Child_A(Base):
    def __init__(self):
        super().__init__()
        
class Child_B(Base):
    def __init__(self):
        Base.__init__(self)
        
Child_A() 
Child_B()
# ---output---
Base created
Base created

Child_A and Child_B can both initialize objects. However, you may wonder what is the difference between super().__init__() and Base.__init__(self)?

In Child_A you get an indirect relation to the __init__() with super(), which uses the class defined to determine the next class's __init__() to locate it in the __mro__.
The other uses the BASE direct, calling the parent class’s method.

Another error you might encounter in MRO is the use of super.

class AA:
    def __init__(self) -> None:
        self.aa = 'aa'
        #super().__init__() without this line, AA won't pass self and execuate __init__() to the next inherit-order
                                             

class BB:
    def __init__(self) -> None:
        super().__init__()
        self.bb = 'bb'

class CC(AA, BB):
    def __init__(self) -> None:
        super().__init__()

dd = CC()
print(CC.__mro__)
print(dd.aa)
print(dd.bb)
---output---
(<class '__main__.DD'>, <class '__main__.AA'>, <class '__main__.BB'>, <class 'object'>)
aa
#AttributeError: 'DD' object has no attribute 'bb'

Look at the class AA. If super().__init__() is not written, it won’t use self go through the next inherit-order, which is class BB. Then it turned out that the CC has no attribute bb. To fix this problem, make sure you add super() in each class that is used in inheritance so that it enters the order in __mro__.

In conclusion, the one used super() has a specific order of parent class by MRO algorithm.
Thesuper() lets you follow the __mro__ to inheritance classes correctly!

How does Python's super() work with multiple inheritance?