Object-oriented Programming in Python — Lesson 5. The limits of inheritance

This is the fifth and last in a series of five articles, summarizing the practical need for object-oriented programming and the facilities provided by object-oriented languages, with examples in Python, stressing the Python interpretation of the paradigm. In the previous chapters, we acquired the basic vocabulary of object-oriented programming: functional substitutability — using the message paradigm — of objects whose behavior and data are encapsulated in classes. Then, based on the observation that applying an object-oriented design in practice must be disciplined, following established idioms, patterns, and architectures, we delved into detailed examples of some complicated — as well as some simple — design patterns. In this final lesson, we discuss the less discussed procedural aspect of object-oriented programming using inheritance, and conclude where we started — substitutability, introducing the famous principle by this name.

Sections in this lesson:

1. The procedural implications of inheritance
2. The “Template Method” Pattern
3. The “Supercall” Idiom
4. The Principle of Substitutability

1. The procedural implications of inheritance

Common programming wisdom tends to emphasize the structural nature of inheritance, especially commonality and variance of data and of its access, as well as the problem-domain semantics of inheritance hierarchies. However, inheritance, and especially when combined with the inevitable functional substitutability, features procedural complexity that is just as interesting and is often overlooked, with dire consequences. A class hierarchy is not only a hierarchy of data. It is also the stratified implementation of one or more algorithms!

From the functional point of view, there are only so many method hierarchies. Classes emerge later. There is some required capability, and then there are so many interesting ways to satisfy it, and then some of these may also inspire more specific ways to satisfy the latter, and so forth. Surprisingly, quite often it turns out that some method hierarchies always go together (for example, they affect the same state), and we unite them into a class hierarchy (complete with the common state).

Inheritance based functional substitutability means the capability to send a message to an object of a specified class and get the response by a method of any of its subclass strata. (And actually, as we shall soon see, possibly also responded partially by methods of its super-class strata, as well as by other methods of its subclass strata!) While — as in any function library implementation — all this complexity is naturally hidden from the sender of the original message, it still requires careful design.

Designing a base class is a non-trivial undertaking. By a base class, you design a framework for others to pour content into, without you knowing exactly what that content will be or how it is going to be used. This requires the following two design responsibilities (at least): (1) to concentrate commonality of control at the appropriate strata, and (2) to facilitate variance of control at the appropriate strata. In this lesson I am going to demonstrate two common techniques: The Template Method Pattern for the first need, and the Supercall idiom for the second.

Let us reconsider the message paradigm. An object responds to messages, applying the appropriate method in its arsenal. (Precisely: applying the appropriate method of its class, using it — the object — as argument). This leads to the — odd, but unavoidable — question: when a message arrives at an object, who responds to it: The object? The object’s inherited functionality? The object’s to be inherited functionality? A combination of both? And consider that while in Python the receiver object’s “superclass (inherited) functionality” and “subclass (to be inherited) functionality” are in the eyes of the beholder (because a Python object relies on exactly one flat attribute dictionary), in the C++ implementation for example, these functional strata are represented by distinct objects!

• “Template method”. Method with “holes”, to be filled in from the actual subclass. The object passes virtual messages to itself, to be overridden in the actual subclasses.
• “Supercall”. Subclass uses inherited method explicitly. Typically, it also adds some actions before and/or after. The most frequent example is the constructor.

2. The “Template Method” Pattern

A “template” method stratifies the responsibility for the job by defining and constraining the procedure within the superclass and allowing (or requiring) some (or all) of the stages in this procedure to be overridden down the inheritance hierarchy.

The following trivial example (this “procedure” does not involve time-ordered stages, but still demonstrates the pattern) is based on the observation that the logic of the capability “to compute bounding rectangle” may be factored out for all Shapes (that is, reduced to a “Template Method” in the abstract base class Shape), provided each particular shape can supply its four corners (according to its own logic).

Output:

((5, 5), (25, 25))

Although the “Template Method” pattern seems trivial, compared with the other design patterns discussed in this series, it has immense design significance. Abstract base classes featuring a Template Method are common in the design of Application Frameworks. Of course, the Template Method pattern’s natural habitat is where stratification of logic is needed (as satisfied by inheritance hierarchy). But where the Framework does not suggest subclassing, straightforward registration of event handlers may do.

3. The “Supercall” Idiom

In contrast with the “Template Method” pattern, a supercall stratifies the responsibility along the inheritance hierarchy by concentrating the process in the subclass and forwarding the stages to its super-classes explicitly (early bound).

In the following example, (1) Circle initialization first allows its Shape stratum to initialize and only then proceeds, and (2) same goes for Circle serialization. These two cases (initialization and serialization) are typical and very frequent in object-oriented software.

Output:

Circle: x:15 y:15 radius:10

4. The Principle of Substitutability

Contrary to some programmers’ intuition, programmatic inheritance, just like the encapsulation on which it is based, is primarily of functionality (methods) and only secondarily of content (data, “state”). Consequently, not every semantic hierarchy that makes sense in the problem domain is also feasible in the programmatic medium, as stated! The fact that something “is a” something else (that is, may be defined in its terms, its cases constitute a subset of the latter’s cases), still does not guarantee that it can successfully subclass it!

Consider the infamous case of Square is a Rectangle. To begin with, we implement the idea of “Rectangle” by a class of this name, storing the Rectangle’s dimensions (height and width), allowing access to this data (the dimensions), and supporting some basic mathematical functionality, such as “to compute area”, “to compute circumference”, “to test intersection (with Point)” and so forth.

A small business that manufactures Rectangles has a quality assurance department that routinely tests the output, by running a simple test on specimens collected at random from the Rectangle product line. This is a simple “sanity test”, testing that the given rectangle, fed with new dimensions, indeed reports the new dimensions. The “sanity” in the check means that this test is so stupid, that there is no reason for it to ever fail (or is there? wait and see!)

Footnotes:

1. Rectangle has two dimensions — height and width — and features the expected getters and setters
2. The product line creates random rectangles.
3. The QA test expects the test to give exactly “10x20”.
4. The test function sets the rectangle to new dimensions and returns a string representation of the new state.

Enter Inheritance (but not triumphantly this time, although it does not know that yet)!

As the demand for Rectangles recedes, the business’ salesperson (who is not a professional Rectangle user) comes up with a brilliant idea: “Squares are in great demand. Since everybody knows that Square is a Rectangle, we can convert our Rectangle product line trivially to manufacture Squares, as well!”. The “is a” association referred to by the salesperson is expressed in “Square is a Rectangle with equal dimensions”, that is, its height is equal to its width.

Apparently, the “Square as Rectangle” proposition is implemented trivially by subclassing. Square needs no additional data. The square’s constructor takes a single dimension and passes it — twice — to its base Rectangle, expressing height = width. To ensure that the “height = width” constraint is always maintained, the dimension setters that were inherited from Rectangle are overridden to set both dimensions.

Footnotes:

1. Rectangle is unaffected by the change.
2. The Square initializes its Rectangle to twice its side.
3. The Square getters return one of the Rectangle dimensions (does not matter which)
4. The setters inherited from Rectangle are overridden to maintain the state of “height = width”.
5. The product line is adjusted to also support Squares (or anything else that is compliant with Rectangle). For the sake of extensibility, the capability to create a single product has been separated to a user supplied function.
6. The task of product creation is delegated to the user-supplied creator.
7. Two creators are provided here: for Rectangle and for Square.
8. The QA test is unaffected by the change. As promised, this is still a Rectangle product line, albeit trivially extended.
9. The standard test is extended to also include Squares.

To the salesperson’s dismay, all shining new squares that emerge from the product line are rejected by Quality Assurance. The sanity check turns out insane: feeding the — for all we know — Rectangle with 10 and 20 for dimensions, we expect the object to report “10x20”, but instead, this lame Rectangle (which happens to be a Square, really) is reporting “20x20”, and must be rejected. Oops!

The are many popular explanations for this counter-intuitive failure. For one, “Rectangle with height = width”, implemented as above, is the invariant of an object, rather than of a class of objects! Suffice to say that — regardless of the problem-domain semantics — programmatically, Square does not inherit Rectangle, or the other way around, or both from a third class, or whatever. Save your breath! Although the above code will compile (in most commercial object-oriented languages, which does not testify in their favor), it features a design bug!

Enter “Design By Contract”!

The “Design By Contract” paradigm (“DBC” for short) was introduced in the 1980’s by Bertrand Meyer, as part of the Eiffel programming language. For our purpose, it responds to the problem of defining and validating function “behavior”, while maintaining — and justifying — the requirement for “information hiding”. Whoever uses an object (that is, sends messages to it) is interested in the job done, caring as little as possible about how the object conducts its business. Information hiding is in the interest of both the client and the supplier of the functionality. But how can the client leave the job to the supplier and wait patiently outside, if the client does not know — and does not want to know — what the supplier is doing in the meantime? DBC reduces the problem to constraints on the state that must prevail before passing control to the method (responsibility of the client) and conditions that must prevail once control passes back (responsibility of the receiver object). The “contract” of a function includes the following clauses:

1. Post-conditions. The result produced, as well as change of state and side effects to this or related objects. The Square in the above example is not Rectangle compliant because the — unspecified but obvious — post-conditions of “to set Rectangle width” (namely: “width = given width and all the rest is unaffected”) is violated by the Square override! Substitutability is about requiring functionality from an object of this — or compatible — type. And that means that the post-conditions of the method that is actually invoked must match those of the method we intended to invoke (in the declared type). (Actually, a limited degree of variance is allowed here, but this is not relevant to the present discussion).
2. Preconditions. The parameters needed, as well as the state of the receiver and related objects. If the preconditions are not met, the post-conditions are undefined (at least. Whether preconditions must be validated is a matter for debate). For example: “Memory copy” preconditions: Target buffer has enough space. Otherwise, the buffer may — or may not — overflow, but the copy is not properly done.
3. Exceptions the method may throw. Situations where the preconditions may be met, but nevertheless the post-conditions may not be guaranteed. “Force majeure”. Recovery, if possible, is the responsibility of the client.

The “contract” of a class may contain, in addition to such clauses per each receivable message, also the invariant of the class: Constraints that must prevail between method invocations — preconditions and post-conditions that need not be stated per message. (But this is not relevant to the present discussion).

Enter the principle of substitutability!

Here is Liskov’s “substitutability” principle (“LSP” for short). Extracted from a lecture given by Barbara Liskov in 1987:

“…If for each object o1 of type S there is an object o2 of type T such that for all programs P defined in terms of T, the behavior of P is unchanged when o1 is substituted for o2 then S is a subtype of T.”

This may seem cryptic, so let us clarify. For practical purposes, we may safely reduce “for all programs P defined in terms of T” to “a function that takes a T”. We may also safely assume that (at least in Python and other contemporary object-oriented languages) “type” is the same with “class” and “subtype” is the same with “subclass” (although this was not the author’s intention). So, now we are talking about a function (or functions) that takes T, and whose “behavior is unchanged” when the T (never mind its private name) is substituted by an S. But just what is meant by “the behavior of P is unchanged”? and how does one define — and test — behavior in the first place? “Behavior” is not a keyword in any programming language that I know of!

It turns out that this (LSP-style) “Behavior” of a method may be easily defined — and tested — in terms of DBC. Code may safely employ a substitutable object (and, for practical reasons, the using function must be aware that the argument is substitutable), when it can have this object replaced by another — which may well be of another type — and it (the code) will still deliver its post-conditions (of course, assuming its pre-conditions are met in the first place).

Returning to the case of “Square as Rectangle”: overriding — in Square — the setters inherited from Rectangle is in violation of LSP. The QA function expects a Rectangle that reports “10x20”, but instead gets an invalid Rectangle (actually Square) that reports “20x20”.

The interesting thing — in the context of the present discussion — about the principle of substitutability is that it uses substitutability of functionality to suggest an inheritance (or some other) type hierarchy, rather than the opposite — and common — paradigm: to first establish an inheritance hierarchy because it seems “obvious” (to someone), and then make the nodes substitutable by inserting functionality and methods to match. (Dogs and Cats are Animals. Why? Because it is obvious! Is there anything that all Animals do? I see. How do Dogs do that and does that differ from the feline way? And so forth). For all we know, in the problem domain it may be possible for one function to consider S as subtype of T and for another, to consider T as subtype of S. Of course, since we expect all functions that take S to also take T, the programmatic solution requires a convenient compromise.

The substitutability principle becomes clear and obvious when one realizes that the whole complexity centers upon one fact that is has never been mentioned, taken for granted: “…and the function uses the argument (to do part of its job)”. In the extreme (but legitimate) case where P does nothing useful with its “substitutable” argument, then, as far as P is concerned, all types are substitutable, which is useless.

The Principle of Substitutability suggests two levels of delegation: (1) the client requires the function to do a job using a (client-provided) argument, and (2) the function relies upon this (client-provided) argument to do part of its own (the function’s) job. If everything goes according to plan (we have substitutability), then the argument will perform its part successfully, causing the function to perform its job successfully!

Consider this business metaphor: We are talking here about a contractor that is capable of doing a job, relying upon a subcontractor to do part of the job, where the subcontractor is provided by the client. If everyone involved is professional and if the pre and post conditions are well defined, then this non-trivial constellation must work, even if the contractor has never met the (client-provided) subcontractor before! For example, even if the contractor is expecting a basic-level hand to be supplied by the client, an over-qualified or irrelevantly specialized hand will still do (because it is still substitutable!)

Morale: We learn that designing software infrastructure for functional substitutability — and especially of the common object-oriented and inheritance-based variety — means furnishing a framework for extension that will (naturally) only work under some strict constraints. And these constraints had better be explicit and formal (so that your potential users — or even yourself — will not abuse them!) There are naturally many pitfalls along the way, but — if one is doing a professional job, as I have tried to demonstrate in this series — these may be easily avoided. Of course, the key to successful design (in any discipline, as well as in software) is to tune the technical tools used in the solution to the real needs posed by the problem. Object oriented programming offers a fair and well-supported (but by no means exclusive) solution to many problems of functional substitutability. But if one starts from the end, forcing an object-oriented design — featuring a rigid inheritance hierarchy and substitutability using messages to anonymous objects — where functional substitutability has never been needed to begin with, just because this is what everyone is doing, then one is practicing bad design, and not because it is — or isn’t — object oriented!

Lessons on this course:

1. Substitutability and Inheritance
2. The glory of OO Substitutability: the “Composite” Pattern
3. The limits of OO substitutability: the “Visitor” Pattern
4. Some boring Design Patterns
5. The limits of inheritance ← (you are here!)