Object-oriented Programming in Python — Lesson 4. Some boring design Patterns

This is the fourth in a series of five lessons, summarizing the practical need for object-oriented programming, and presenting the common facilities provided by object-oriented languages. The examples are in Python, stressing the Python implementation and approach. In the previous chapters, we acquired the basic vocabulary of object-oriented programming: functional substitutability — using the message paradigm — of objects whose behavior and data are encapsulated in classes. Then, based on the observation that applying an object-oriented design in practice must be disciplined, following established idioms, patterns, and architectures, we delved into detailed examples of some “heavy-weight” polymorphic design pattern applications, centered on recursive data structures. This fourth lesson, in comparison, proceeds on a low key, to consider some minor design patterns that may be needed on almost daily basis and their implementation is simplified in a dynamic “duck-typed” language like Python.

Sections in this Lesson:

1. “Dynamic Pluggable Factory” pattern language, by type (“Factory-Method” based)
2. “Dynamic Pluggable Factory” pattern language, by example (“Prototype” based)
3. The “Visitor” Pattern, revisited.

1. “Dynamic Pluggable Factory”, by type

A complete design pattern merges two perspectives. (1) from the requirements perspective, a design pattern is a response to a problem in a context. (2) from the implementation perspective, it is a pattern (as the name suggests) for arranging items in space that suggests functionality (to the potential user). (I am using the terms “items” and “space” — instead of “code constructs” and “program” — to respect the original design patterns of building architecture, which inspired our design patterns of software). And on top of this, there is one crucial constraint: “Repeated a million times, it (the design pattern implementation) will never be the same!” Namely, an honest design pattern may never be reduced to a one-size-fits-all reusable artifact, such as function, class template, base class, etc.

Nevertheless, the sad truth is that, while the latter constraint is indeed true for some structural and polymorphic patterns of software design (such as the “Composite” Pattern, discussed in Chapter 2), some of the canonical design patterns of software barely live up to it. Some minor design patterns may — and have been — automated. And some of the problems that some design patterns of software respond to are platform-dependent and self-inflicted.

The “Dynamic Pluggable Factory” is a pattern language involving three patterns — Factory, Singleton, and Factory Method — as well as the idioms of Registration and of mapping keyword to functionality.

For example, consider a Message Factory, which enables the application to receive messages polymorphically (that is, without explicitly knowing exactly what is received, and there being multiple and unlimited alternatives). The key to the complexity of this problem is that in the object-oriented paradigm, “receiving” such message must involve creating an autonomous, encapsulated message object, which further requires to predefine a message class, and more than one of concrete subclasses of it (considering the structural — and possibly functional — diversity of message types). This non-trivial architecture does not point to a deficiency of the object-oriented paradigm. The complexity is inherent in the problem domain. The object-oriented interpretation does justice to the problem, by explicating and formalizing its complexity, facilitating precision treatment. (Compare this with the alternative of ignoring the complexity and — inevitably — wasting so much time on finding it out by trial and error). So, the Message Factory is capable to silently create a message of the correct Message type, given message-id and raw message block.

Enter the “Registration” idiom! (Just “idiom” — it is not cool enough to deserve the title of “Design Pattern”).

The design of the Message Factory is open (for extension). The use of registration of message creators per message-id removes all knowledge of the available concrete Message types from the Factory. On the contrary, compare this solution with the archetypal alternative of a “switch/case” mechanism, where all message types — as well as the raw message layouts and the capability to serialize them from the block — must be known in advance and explicit in the usage context. While this alternative remains valid for a small and finite repertoire of message types, such as in message-id enumeration, it is not useful in real-world messaging systems, where message types (and their processing) are typically added on the fly, and by a third party. Consequently, the last thing we may need here is to hard code the repertoire of message types, their layout, and their creation procedure!

The term “Dynamic Pluggable Factory” was coined by John Vlissides (of “Gang of Four” fame) in an article in the Journal of Object-Oriented Programming, elaborating on the canonical Abstract Factory Pattern, by introducing two additional qualities: (1) it is dynamic (featuring the capability to add subject types to create) and (2) pluggable (the types to create, as well as their creation procedure, accumulate from the outside).

Here is a summary of Dynamic Pluggable Factory functionality:

The Message Factory Maps message-id to “the ability to create message (given raw message block)”. In particular:

1. The Message Factory is capable “to create a Message”, using message-id and message block, giving Message object.
2. The Message Factory is capable “to register Message creator”, using message-id and an (anonymous) message creator. The Message Factory is oblivious to the types of both message creator and the message (that that the latter creates).

Enter the “Singleton”!

And where does one find this Message Factory? Obviously, our Message Factory must be a Singleton — another design pattern. Being a Singleton means that (1) there is only one such, and (2) it is always accessible and ready (when needed). Consider what chaos will be caused by an implementation that allows multiple Message Factories (with different population of message creators in each). Obviously, in a sane universe, there must be exactly one Message Factory, and it must be global and available to all.

A “Singleton” entity encapsulates a globally available resource. In the strict (pre-Pythonic) object-oriented interpretation, it used to be implemented as a class that was guaranteed to have a single instance. Why go to this length — class, instance — rather than just allocate a global dictionary object? While a global variable is indeed available globally, we cannot guarantee (in many programming languages) that it will be fully initialized and available for service when required (for example, by another global object, during initialization time).

Contrary to some people’s intuition, it is the second quality (being ready when required) — rather than just the first (only one such) that makes a Singleton! For example, consider The Application in an application-framework-based design. There is only one Application. But the singularity of The Application (only one such) still does not make it a Singleton! There is no need here for a guaranteed readiness mechanism because no one is addressing The Application from the outside. On the contrary, it is The Application that addresses everyone else from the inside. And there is no need to create it on demand. The (one) Application is created statically at program entry-point, which is good enough.

Here is a summary of the Singleton functionality:

1. A “Singleton” is implemented as a class that is restricted (procedurally) to a single instance.
2. The Singleton class is capable to reveal the Singleton instance. (Yes, the class is itself an object, featuring its own methods!)
3. The singleton instance — a hidden member of the class (rather than any object) — is initialized by the first access request (normally, by testing if it does not yet exist).
4. In addition, the Singleton instance may be made thread-safe where necessary (avoiding concurrent initialization, which may result in the accidental proliferation of multiple “Singleton” instances, with only one of them available globally).

Enter the “Factory Method”!

And then, what exactly does one register in the Message Factory? What magic is hidden behind the required functionality of message creator? The last pattern to complete the pattern language is the “Factory Method”. A Factory Method is capable to create an object of some type and only that type (and the type is not known to the client). Being in possession of a Factory Method gives the client the capability to create objects of the appropriate type, while being exempt from knowing what that type is. (Which may be complicated to explain but is a legitimate requirement in many programmatic contexts). Obviously, Factory Methods naturally fit the role of “creators” in a Dynamic Pluggable Factory.

Quite inconveniently, in the canonical Factory Method solution, catering to strong-typed languages, the required factory methods must form a class hierarchy parallel with the subject (for example, Message) hierarchy. While, in the object-oriented interpretation, each message must be implemented as a class, and all these classes must derive from the abstract class Message, we must add on top of this, a parallel hierarchy of Message Creators, one for each.

This is cumbersome, as stated! However, where genericity is supported (for example, in C++), with the appropriate class-template infrastructure, defining a specific Message Creator may be reduced to a single macro invocation. So this takes the complexity away (from the user of the API).

Here is a (rather forced) attempt to render a Dynamic-Pluggable Message Factory “by the book” in Python:

Footnotes:

1. All messages start with the same header.
2. Message id and size are to be received from the owner message.
3. The data size is for used by a third party (not used in this example).
4. For the sake of simplicity, the message is encoded as plain text with fixed-size fields.
5. The abstract base Message takes care only of header. The data is subclass specific.
6. First the header is serialized, then the data. The buffer is (in this example) created by the Message.
7. First the header is deserialized, then the data.
8. Data serialization is subclass specific.
9. Data deserialization is subclass specific.
10. A message creator can create some Message.
11. Get-id is used for registration.
12. The singleton instance hides in the class.
13. Factory-method creators register by message-id.
14. Message creation is silently delegated to the registered creator.
15. The singleton instance is created on demand. This implementation does not expect concurrency.
16. The message queue stores message buffers (rather than objects!)
17. The message to be stored is deserialized.
18. The message-id is extracted from the stored header.
19. The appropriate message is created by the singleton message factory, using message-id.
20. The empty message is serialized from the buffer.

One does not have to be a Python expert in order to make the obvious observation that this design begs to be simplified and stripped of some redundant detail. Which leads us back to the observation that some of the minor problems that this pattern language originally came to solve were platform-dependent and self-inflicted! (Without negating the validity and usefulness of the overall Dynamic-Pluggable Factory problem itself!)

Especially, the following Python features could be useful here:

1. The availability of modules. A Python module trivially fulfills both requirements from a Singleton. (1) there is only one such (the import request is not executed twice). And (2) the module is guaranteed to be fully initialized when addressed from another module (there must have been an import request prior to this), (except for recursive import-loop, which is invalid anyway).
2. One may either consider the entire module as the Singleton and address its global functions or address an object that exists inside the module — both methods will work. In any case, there is no need to request anyone to reveal (and create) the instance!
3. The availability of the class object in Python. When all there is to object creation is bare initialization (using the provided particular Message constructor), then the class object will suffice. There is seldom a practical need to go through the trouble of defining a parallel creator hierarchy.

Here is a “native” Python rendering of the dynamic-pluggable factory:

Footnotes:

1. The Message and its header are unaffected by the change.
2. The (singleton) message factory is reduced to a dictionary object.
3. Incoming Message is initialized by its type, registered in the factory object.
4. Message creator registration is native type based. No user-defined creators used.

3. “Dynamic Pluggable Factory”, by example

The Factory-Method-based Dynamic Pluggable Factory addresses the need to map message-ids to Message objects. Consequently, it is useful when there is such one-to-one correlation between the input (type hint) and the output (some object of that type), and when the object is created by default. But it fails to address the need to create objects that, although determined by the input hint, are not necessarily of diverse types! We may be required to use a factory to populate our world by example: with objects that are determined by content as well as by type!

An example of the latter need is a GUI factory (dynamic and pluggable), used to create shapes visually, and that means not (necessarily) by type, but by content as well. The creation hint here will be an icon — a visible example. The user of the graphic editor drags (duplicate) objects to the canvas from the toolbox. In addition, the user may also select a shape from the canvas and drag it to the toolbox, thus adding another icon for selection.

In the object-oriented interpretation, this example must be backed by an object of the appropriate type and content! There is no need to develop a factory method for this job. The operational objects are already equipped with all that it takes.

In this solution, the factory registers objects (of the same type to be created), typically responding to clone, which returns a copy (“clone”) of the object.

4. The “Visitor” Pattern revisited

Other design patterns that may be simplified in a duck-typed language like Python, are heavily polymorphic patterns like the “Visitor” Pattern, where substitutability relies on intensive inheritance hierarchies. For example, here is a reduced and simplified version of the “Visitor” Pattern (discussed at large in Chapter 3 of this series), concentrating on the accept/visit loop.

But first, a short reminder. Many manipulations are possible over Shapes, which for some reason, must be carried from the outside. Although these capabilities involve Shape data and possibly intrinsic Shape logic, they may not be methods of Shape, because they are equally dependent upon other machinery and infrastructure, which are not Shape-specific and are configurable. For example, to display Shape. While this capability certainly relies upon obtaining the Shape’s dimensions and knowing what to do with them (for example, a Circle’s radius), it also involves the type of display technology (vector or pixel graphics), screen resolution, magnification, aspect ratio, the use of fonts, and so forth, which have nothing to with Shapes and may be reconfigured at will.

This problem domain features double polymorphism — both the type of Shape and the type of its manipulator are substitutable. But it is asymmetric. The manipulators need the Shapes. In fact, all manipulator types need to know all Shape types in advance (to manipulate each one of them). But the Shapes are not concerned with the manipulator repertoire, which for all they know, is indefinite. They just grant access to the manipulator — called here “Visitor” — whatever it may be.

There is more than one established solution to this problem. Here we apply the traditional object oriented “Visitor” Pattern, which uses the idiom of double dispatch. Each request from a Visitor to do something to a Shape involves two polymorphic messages: from Visitor to Shape and back from Shape to Visitor.

To begin with, here is a (rather forced) solution “by the book”, featuring a polymorphic implementation of the accept method.

1. A Shape (which is interface for any Shape), is capable to accept a “Shape Visitor”, for whatever manipulation the latter is capable of. Note that “Shape Visitor” is a forward declaration! (The class is defined later on).
2. The concrete Circle implementation of accept directs the visitor to visit this object, revealing its true nature as a Circle.
3. “Shape Visitor” is interface for any Shape manipulator, capable to visit each known Shape type. (And all Shape types must, therefore, be known in advance!) The method is subclass specific.
4. “Shape Painter” is a concrete Shape Visitor, featuring concrete visit methods for all known Shape types.
5. The Shape Painter is requested to draw an anonymous Shape (for example, one in a row). It does not have to know the true type of the Shape to do the job (or does it?). It requests the Shape, whatever it is, to accept it, which will eventually end up in the Shape Painter requested to visit the particular Shape. (Because the Shape knows its own type!)
6. A Shape Painter, created on the fly, is requested to draw a Shape — actually a Circle, also created on the fly.

The accept method hierarchy is clearly redundant. All accept methods are identical, with the only difference being the Shape-type part of the name. Using Python’s easy access to metadata, we can concentrate the Visitor-method selection capability in the base Shape, as follows:

Footnotes:

1. Although Shape is meant to be just an interface, it has no abstract methods and therefore, there is no point in subclassing ABC here.
2. accept is a concrete method of the base Shape, which is not meant to be overridden by subclass Shapes.
3. Inside the common accept the desired Visitor Method name is constructed (from the prefix “visit” and the actual class name). Although the method is “hard-coded” in the base Shape, during runtime it is going to retrieve the class of the actual Shape responding to the message, for example, Circle. We can be sure that it will never be Shape because we do not intend to create such objects.
4. This solution does not require all Shape types to be known in advance, risking the possibility of attempting to visit an unsupported Shape!
5. The common accept method retrieves the appropriate function (by just-constructed name) from this object’s dictionary and activates it using this object.
6. In this solution, concrete Shapes do not override accpet!
7. The Shape Visitor and its implementation are not affected by the change.

Finally, given this simple name-based runtime dispatch, the whole double-dispatch strategy becomes redundant! If we remove the dispatch mechanism from Shape and put it in the base Visitor, we end up with a Shape hierarchy that is oblivious to the existence of Visitors and a Shape Visitor hierarchy that is concerned with discovering its own methods. This is a simple down-to-earth object-oriented solution, where every object is restricted to its own responsibility, removing the cross-hierarchy double dispatch noise from the problem domain. We still have double dispatch (we reach the desired method in two invocations), but it is hidden in the Visitor hierarchy.

Footnotes:

1. Shape is innocent of all knowledge of the Visitor hierarchy.
2. The dispatch is now located at the abstract base Visitor.

5. What next?

In the first Four lessons, we were introduced to the object-oriented solution to the design challenge of functional substitutability: the Message paradigm, involving passing a message to an object of unknown type, resolved during runtime by a method (by that name) in its class. We have seen how Python’s “duck typing” makes substitutability intuitive among any informally — but semantically — related types. To stress the importance of conducting object-oriented design and programming according to established practices we went through some major design patterns, such as Composite and Visitor. Then, we considered some minor design patterns such as Dynamic Pluggable Factory, Singleton and Prototype, taking advantage of Python’s convenient access to metadata to simplify their implementation. In the remaining lesson, we are going to consider the un-celebrated procedural aspect of the inheritance hierarchy, summarize the design opportunities enabled by the Python object-oriented infrastructure, and finalize with the same topic with which we started — and which has been the backbone of this entire series — substitutability, discussing the principle of substitutability.

Lessons on this course:

1. Substitutability and Inheritance
2. The glory of OO Substitutability: the “Composite” Pattern
3. The limits of OO substitutability: the “Visitor” Pattern
4. Some boring Design Patterns ← (you are here!)
5. The limits of inheritance